Beruflich Dokumente
Kultur Dokumente
Stephen C. Johnson
ABSTRACT
Computer program input generally has some structure; in fact, every computer pro-
gram that does input can be thought of as defining an ‘‘input language’’ which it accepts.
An input language may be as complex as a programming language, or as simple as a
sequence of numbers. Unfortunately, usual input facilities are limited, difficult to use,
and often are lax about checking their inputs for validity.
Yacc provides a general tool for describing the input to a computer program. The
Yacc user specifies the structures of his input, together with code to be invoked as each
such structure is recognized. Yacc turns such a specification into a subroutine that han-
dles the input process; frequently, it is convenient and appropriate to have most of the
flow of control in the user’s application handled by this subroutine.
The input subroutine produced by Yacc calls a user-supplied routine to return the
next basic input item. Thus, the user can specify his input in terms of individual input
characters, or in terms of higher level constructs such as names and numbers. The user-
supplied routine may also handle idiomatic features such as comment and continuation
conventions, which typically defy easy grammatical specification.
Yacc is written in portable C. The class of specifications accepted is a very general
one: LALR(1) grammars with disambiguating rules.
In addition to compilers for C, APL, Pascal, RATFOR, etc., Yacc has also been
used for less conventional languages, including a phototypesetter language, several desk
calculator languages, a document retrieval system, and a Fortran debugging system.
0: Introduction
Yacc provides a general tool for imposing structure on the input to a computer program. The Yacc
user prepares a specification of the input process; this includes rules describing the input structure, code to
be invoked when these rules are recognized, and a low-level routine to do the basic input. Yacc then gen-
erates a function to control the input process. This function, called a parser , calls the user-supplied low-
level input routine (the lexical analyzer ) to pick up the basic items (called tokens ) from the input stream.
These tokens are organized according to the input structure rules, called grammar rules ; when one of these
rules has been recognized, then user code supplied for this rule, an action , is invoked; actions have the
ability to return values and make use of the values of other actions.
Yacc is written in a portable dialect of C1 and the actions, and output subroutine, are in C as well.
Moreover, many of the syntactic conventions of Yacc follow C.
The heart of the input specification is a collection of grammar rules. Each rule describes an allow-
able structure and gives it a name. For example, one grammar rule might be
date : monthiname day ´,´ year ;
Here, date , monthiname , day , and year represent structures of interest in the input process; presumably,
monthiname , day , and year are defined elsewhere. The comma ‘‘,’’ is enclosed in single quotes; this
implies that the comma is to appear literally in the input. The colon and semicolon merely serve as punc-
tuation in the rule, and have no significance in controlling the input. Thus, with proper definitions, the
-- --
input
July 4, 1776
might be matched by the above rule.
An important part of the input process is carried out by the lexical analyzer. This user routine reads
the input stream, recognizing the lower level structures, and communicates these tokens to the parser. For
historical reasons, a structure recognized by the lexical analyzer is called a terminal symbol , while the
structure recognized by the parser is called a nonterminal symbol . To avoid confusion, terminal symbols
will usually be referred to as tokens .
There is considerable leeway in deciding whether to recognize structures using the lexical analyzer
or grammar rules. For example, the rules
monthiname : ´J´ ´a´ ´n´ ;
monthiname : ´F´ ´e´ ´b´ ;
...
The next several sections describe the basic process of preparing a Yacc specification; Section 1
describes the preparation of grammar rules, Section 2 the preparation of the user supplied actions associ-
ated with these rules, and Section 3 the preparation of lexical analyzers. Section 4 describes the operation
of the parser. Section 5 discusses various reasons why Yacc may be unable to produce a parser from a
specification, and what to do about it. Section 6 describes a simple mechanism for handling operator pre-
cedences in arithmetic expressions. Section 7 discusses error detection and recovery. Section 8 discusses
the operating environment and special features of the parsers Yacc produces. Section 9 gives some sugges-
tions which should improve the style and efficiency of the specifications. Section 10 discusses some
advanced topics, and Section 11 gives acknowledgements. Appendix A has a brief example, and Appendix
B gives a summary of the Yacc input syntax. Appendix C gives an example using some of the more
advanced features of Yacc, and, finally, Appendix D describes mechanisms and syntax no longer actively
supported, but provided for historical continuity with older versions of Yacc.
1: Basic Specifications
Names refer to either tokens or nonterminal symbols. Yacc requires token names to be declared as
such. In addition, for reasons discussed in Section 3, it is often desirable to include the lexical analyzer as
part of the specification file; it may be useful to include other programs as well. Thus, every specification
file consists of three sections: the declarations , (grammar) rules , and programs . The sections are
separated by double percent ‘‘%%’’ marks. (The percent ‘‘%’’ is generally used in Yacc specifications as
an escape character.)
In other words, a full specification file looks like
declarations
%%
rules
%%
programs
The declaration section may be empty. Moreover, if the programs section is omitted, the second %%
mark may be omitted also; thus, the smallest legal Yacc specification is
%%
rules
Blanks, tabs, and newlines are ignored except that they may not appear in names or multi-character
reserved symbols. Comments may appear wherever a name is legal; they are enclosed in /∗ . . . ∗/, as in C
and PL/I.
The rules section is made up of one or more grammar rules. A grammar rule has the form:
A : BODY ;
A represents a nonterminal name, and BODY represents a sequence of zero or more names and literals.
The colon and the semicolon are Yacc punctuation.
Names may be of arbitrary length, and may be made up of letters, dot ‘‘.’’, underscore ‘‘i’’, and
non-initial digits. Upper and lower case letters are distinct. The names used in the body of a grammar rule
may represent tokens or nonterminal symbols.
A literal consists of a character enclosed in single quotes ‘‘´’’. As in C, the backslash ‘‘\’’ is an
escape character within literals, and all the C escapes are recognized. Thus
-- --
´\n´ newline
´\r´ return
´\´´ single quote ‘‘´’’
´\\´ backslash ‘‘\’’
´\t´ tab
´\b´ backspace
´\f´ form feed
´\xxx´‘‘xxx’’ in octal
For a number of technical reasons, the NUL character (´\0´ or 0) should never be used in grammar rules.
If there are several grammar rules with the same left hand side, the vertical bar ‘‘|’’ can be used to
avoid rewriting the left hand side. In addition, the semicolon at the end of a rule can be dropped before a
vertical bar. Thus the grammar rules
A : B C D ;
A : E F ;
A : G ;
can be given to Yacc as
A : B C D
| E F
| G
;
It is not necessary that all grammar rules with the same left side appear together in the grammar rules sec-
tion, although it makes the input much more readable, and easier to change.
If a nonterminal symbol matches the empty string, this can be indicated in the obvious way:
empty : ;
Names representing tokens must be declared; this is most simply done by writing
%token name1 name2 . . .
in the declarations section. (See Sections 3 , 5, and 6 for much more discussion). Every name not defined
in the declarations section is assumed to represent a nonterminal symbol. Every nonterminal symbol must
appear on the left side of at least one rule.
Of all the nonterminal symbols, one, called the start symbol , has particular importance. The parser
is designed to recognize the start symbol; thus, this symbol represents the largest, most general structure
described by the grammar rules. By default, the start symbol is taken to be the left hand side of the first
grammar rule in the rules section. It is possible, and in fact desirable, to declare the start symbol explicitly
in the declarations section using the %start keyword:
%start symbol
The end of the input to the parser is signaled by a special token, called the endmarker . If the tokens
up to, but not including, the endmarker form a structure which matches the start symbol, the parser function
returns to its caller after the endmarker is seen; it accepts the input. If the endmarker is seen in any other
context, it is an error.
It is the job of the user-supplied lexical analyzer to return the endmarker when appropriate; see sec-
tion 3, below. Usually the endmarker represents some reasonably obvious I/O status, such as ‘‘end-of-
file’’ or ‘‘end-of-record’’.
2: Actions
With each grammar rule, the user may associate actions to be performed each time the rule is recog-
nized in the input process. These actions may return values, and may obtain the values returned by
-- --
previous actions. Moreover, the lexical analyzer can return values for tokens, if desired.
An action is an arbitrary C statement, and as such can do input and output, call subprograms, and
alter external vectors and variables. An action is specified by one or more statements, enclosed in curly
braces ‘‘{’’ and ‘‘}’’. For example,
A : ´(´ B ´)´
{ hello( 1, "abc" ); }
and
XXX : YYY ZZZ
{ printf("a message\n");
flag = 25; }
are grammar rules with actions.
To facilitate easy communication between the actions and the parser, the action statements are
altered slightly. The symbol ‘‘dollar sign’’ ‘‘$’’ is used as a signal to Yacc in this context.
To return a value, the action normally sets the pseudo-variable ‘‘$$’’ to some value. For example,
an action that does nothing but return the value 1 is
{ $$ = 1; }
To obtain the values returned by previous actions and the lexical analyzer, the action may use the
pseudo-variables $1, $2, . . ., which refer to the values returned by the components of the right side of a
rule, reading from left to right. Thus, if the rule is
A : B C D ;
for example, then $2 has the value returned by C, and $3 the value returned by D.
As a more concrete example, consider the rule
expr : ´(´ expr ´)´ ;
The value returned by this rule is usually the value of the expr in parentheses. This can be indicated by
expr : ´(´ expr ´)´ { $$ = $2 ; }
By default, the value of a rule is the value of the first element in it ($1). Thus, grammar rules of the
form
A : B ;
frequently need not have an explicit action.
In the examples above, all the actions came at the end of their rules. Sometimes, it is desirable to get
control before a rule is fully parsed. Yacc permits an action to be written in the middle of a rule as well as
at the end. This rule is assumed to return a value, accessible through the usual mechanism by the actions to
the right of it. In turn, it may access the values returned by the symbols to its left. Thus, in the rule
A : B
{ $$ = 1; }
C
{ x = $2; y = $3; }
;
the effect is to set x to 1, and y to the value returned by C.
Actions that do not terminate a rule are actually handled by Yacc by manufacturing a new nontermi-
nal symbol name, and a new rule matching this name to the empty string. The interior action is the action
triggered off by recognizing this added rule. Yacc actually treats the above example as if it had been writ-
ten:
-- --
$ACT : /∗ empty ∗/
{ $$ = 1; }
;
A : B $ACT C
{ x = $2; y = $3; }
;
In many applications, output is not done directly by the actions; rather, a data structure, such as a
parse tree, is constructed in memory, and transformations are applied to it before output is generated. Parse
trees are particularly easy to construct, given routines to build and maintain the tree structure desired. For
example, suppose there is a C function node , written so that the call
node( L, n1, n2 )
creates a node with label L, and descendants n1 and n2, and returns the index of the newly created node.
Then parse tree can be built by supplying actions such as:
expr : expr ´+´ expr
{ $$ = node( ´+´, $1, $3 ); }
in the specification.
The user may define other variables to be used by the actions. Declarations and definitions can
appear in the declarations section, enclosed in the marks ‘‘%{’’ and ‘‘%}’’. These declarations and
definitions have global scope, so they are known to the action statements and the lexical analyzer. For
example,
%{ int variable = 0; %}
could be placed in the declarations section, making variable accessible to all of the actions. The Yacc
parser uses only names beginning in ‘‘yy’’; the user should avoid such names.
In these examples, all the values are integers: a discussion of values of other types will be found in
Section 10.
3: Lexical Analysis
The user must supply a lexical analyzer to read the input stream and communicate tokens (with
values, if desired) to the parser. The lexical analyzer is an integer-valued function called yylex . The func-
tion returns an integer, the token number , representing the kind of token read. If there is a value associated
with that token, it should be assigned to the external variable yylval .
The parser and the lexical analyzer must agree on these token numbers in order for communication
between them to take place. The numbers may be chosen by Yacc, or chosen by the user. In either case,
the ‘‘# define’’ mechanism of C is used to allow the lexical analyzer to return these numbers symbolically.
For example, suppose that the token name DIGIT has been defined in the declarations section of the Yacc
specification file. The relevant portion of the lexical analyzer might look like:
-- --
yylex(){
extern int yylval;
int c;
...
c = getchar();
...
switch( c ) {
...
case ´0´:
case ´1´:
...
case ´9´:
yylval = c−´0´;
return( DIGIT );
...
}
...
The intent is to return a token number of DIGIT, and a value equal to the numerical value of the
digit. Provided that the lexical analyzer code is placed in the programs section of the specification file, the
identifier DIGIT will be defined as the token number associated with the token DIGIT.
This mechanism leads to clear, easily modified lexical analyzers; the only pitfall is the need to avoid
using any token names in the grammar that are reserved or significant in C or the parser; for example, the
use of token names if or while will almost certainly cause severe difficulties when the lexical analyzer is
compiled. The token name error is reserved for error handling, and should not be used naively (see Sec-
tion 7).
As mentioned above, the token numbers may be chosen by Yacc or by the user. In the default situa-
tion, the numbers are chosen by Yacc. The default token number for a literal character is the numerical
value of the character in the local character set. Other names are assigned token numbers starting at 257.
To assign a token number to a token (including literals), the first appearance of the token name or
literal in the declarations section can be immediately followed by a nonnegative integer. This integer is
taken to be the token number of the name or literal. Names and literals not defined by this mechanism
retain their default definition. It is important that all token numbers be distinct.
For historical reasons, the endmarker must have token number 0 or negative. This token number
cannot be redefined by the user; thus, all lexical analyzers should be prepared to return 0 or negative as a
token number upon reaching the end of their input.
A very useful tool for constructing lexical analyzers is the Lex program developed by Mike Lesk.8
These lexical analyzers are designed to work in close harmony with Yacc parsers. The specifications for
these lexical analyzers use regular expressions instead of grammar rules. Lex can be easily used to pro-
duce quite complicated lexical analyzers, but there remain some languages (such as FORTRAN) which do
not fit any theoretical framework, and whose lexical analyzers must be crafted by hand.
The machine has only four actions available to it, called shift , reduce , accept , and error . A move
of the parser is done as follows:
1. Based on its current state, the parser decides whether it needs a lookahead token to decide what
action should be done; if it needs one, and does not have one, it calls yylex to obtain the next token.
2. Using the current state, and the lookahead token if needed, the parser decides on its next action, and
carries it out. This may result in states being pushed onto the stack, or popped off of the stack, and in
the lookahead token being processed or left alone.
The shift action is the most common action the parser takes. Whenever a shift action is taken, there
is always a lookahead token. For example, in state 56 there may be an action:
IF shift 34
which says, in state 56, if the lookahead token is IF, the current state (56) is pushed down on the stack, and
state 34 becomes the current state (on the top of the stack). The lookahead token is cleared.
The reduce action keeps the stack from growing without bounds. Reduce actions are appropriate
when the parser has seen the right hand side of a grammar rule, and is prepared to announce that it has seen
an instance of the rule, replacing the right hand side by the left hand side. It may be necessary to consult
the lookahead token to decide whether to reduce, but usually it is not; in fact, the default action
(represented by a ‘‘.’’) is often a reduce action.
Reduce actions are associated with individual grammar rules. Grammar rules are also given small
integer numbers, leading to some confusion. The action
. reduce 18
refers to grammar rule 18, while the action
IF shift 34
refers to state 34.
Suppose the rule being reduced is
A : x y z ;
The reduce action depends on the left hand symbol (A in this case), and the number of symbols on the right
hand side (three in this case). To reduce, first pop off the top three states from the stack (In general, the
number of states popped equals the number of symbols on the right side of the rule). In effect, these states
were the ones put on the stack while recognizing x , y , and z , and no longer serve any useful purpose.
After popping these states, a state is uncovered which was the state the parser was in before beginning to
process the rule. Using this uncovered state, and the symbol on the left side of the rule, perform what is in
effect a shift of A. A new state is obtained, pushed onto the stack, and parsing continues. There are
significant differences between the processing of the left hand symbol and an ordinary shift of a token,
however, so this action is called a goto action. In particular, the lookahead token is cleared by a shift, and
is not affected by a goto. In any case, the uncovered state contains an entry such as:
A goto 20
causing state 20 to be pushed onto the stack, and become the current state.
In effect, the reduce action ‘‘turns back the clock’’ in the parse, popping the states off the stack to go
back to the state where the right hand side of the rule was first seen. The parser then behaves as if it had
seen the left side at that time. If the right hand side of the rule is empty, no states are popped off of the
stack: the uncovered state is in fact the current state.
The reduce action is also important in the treatment of user-supplied actions and values. When a rule
is reduced, the code supplied with the rule is executed before the stack is adjusted. In addition to the stack
holding the states, another stack, running in parallel with it, holds the values returned from the lexical
analyzer and the actions. When a shift takes place, the external variable yylval is copied onto the value
stack. After the return from the user code, the reduction is carried out. When the goto action is done, the
external variable yyval is copied onto the value stack. The pseudo-variables $1, $2, etc., refer to the value
-- --
stack.
The other two parser actions are conceptually much simpler. The accept action indicates that the
entire input has been seen and that it matches the specification. This action appears only when the looka-
head token is the endmarker, and indicates that the parser has successfully done its job. The error action,
on the other hand, represents a place where the parser can no longer continue parsing according to the
specification. The input tokens it has seen, together with the lookahead token, cannot be followed by any-
thing that would result in a legal input. The parser reports an error, and attempts to recover the situation
and resume parsing: the error recovery (as opposed to the detection of error) will be covered in Section 7.
It is time for an example! Consider the specification
%token DING DONG DELL
%%
rhyme : sound place
;
sound : DING DONG
;
place : DELL
;
When Yacc is invoked with the −v option, a file called y.output is produced, with a human-readable
description of the parser. The y.output file corresponding to the above grammar (with some statistics
stripped off the end) is:
-- --
state 0
$accept : irhyme $end
DING shift 3
. error
rhyme goto 1
sound goto 2
state 1
$accept : rhymei$end
$end accept
. error
state 2
rhyme : soundiplace
DELL shift 5
. error
place goto 4
state 3
sound : DINGiDONG
DONG shift 6
. error
state 4
rhyme : sound placei (1)
. reduce 1
state 5
place : DELLi (3)
. reduce 3
state 6
sound : DING DONGi (2)
. reduce 2
Notice that, in addition to the actions for each state, there is a description of the parsing rules being pro-
cessed in each state. The i character is used to indicate what has been seen, and what is yet to come, in
each rule. Suppose the input is
DING DONG DELL
It is instructive to follow the steps of the parser while processing this input.
Initially, the current state is state 0. The parser needs to refer to the input in order to decide between
the actions available in state 0, so the first token, DING , is read, becoming the lookahead token. The
action in state 0 on DING is is ‘‘shift 3’’, so state 3 is pushed onto the stack, and the lookahead token is
cleared. State 3 becomes the current state. The next token, DONG , is read, becoming the lookahead
-- --
token. The action in state 3 on the token DONG is ‘‘shift 6’’, so state 6 is pushed onto the stack, and the
lookahead is cleared. The stack now contains 0, 3, and 6. In state 6, without even consulting the looka-
head, the parser reduces by rule 2.
sound : DING DONG
This rule has two symbols on the right hand side, so two states, 6 and 3, are popped off of the stack, uncov-
ering state 0. Consulting the description of state 0, looking for a goto on sound ,
sound goto 2
is obtained; thus state 2 is pushed onto the stack, becoming the current state.
In state 2, the next token, DELL , must be read. The action is ‘‘shift 5’’, so state 5 is pushed onto the
stack, which now has 0, 2, and 5 on it, and the lookahead token is cleared. In state 5, the only action is to
reduce by rule 3. This has one symbol on the right hand side, so one state, 5, is popped off, and state 2 is
uncovered. The goto in state 2 on place , the left side of rule 3, is state 4. Now, the stack contains 0, 2, and
4. In state 4, the only action is to reduce by rule 1. There are two symbols on the right, so the top two
states are popped off, uncovering state 0 again. In state 0, there is a goto on rhyme causing the parser to
enter state 1. In state 1, the input is read; the endmarker is obtained, indicated by ‘‘$end’’ in the y.output
file. The action in state 1 when the endmarker is seen is to accept, successfully ending the parse.
The reader is urged to consider how the parser works when confronted with such incorrect strings as
DING DONG DONG , DING DONG , DING DONG DELL DELL , etc. A few minutes spend with this and
other simple examples will probably be repaid when problems arise in more complicated contexts.
IF ( C1 ) {
IF ( C2 ) S1
}
ELSE S2
or
IF ( C1 ) {
IF ( C2 ) S1
ELSE S2
}
The second interpretation is the one given in most programming languages having this construct. Each
ELSE is associated with the last preceding ‘‘un-ELSE’d’’ IF . In this example, consider the situation where
the parser has seen
IF ( C1 ) IF ( C2 ) S1
and is looking at the ELSE . It can immediately reduce by the simple-if rule to get
IF ( C1 ) stat
and then read the remaining input,
ELSE S2
and reduce
IF ( C1 ) stat ELSE S2
by the if-else rule. This leads to the first of the above groupings of the input.
On the other hand, the ELSE may be shifted, S2 read, and then the right hand portion of
IF ( C1 ) IF ( C2 ) S1 ELSE S2
can be reduced by the if-else rule to get
IF ( C1 ) stat
which can be reduced by the simple-if rule. This leads to the second of the above groupings of the input,
which is usually desired.
Once again the parser can do two valid things − there is a shift/reduce conflict. The application of
disambiguating rule 1 tells the parser to shift in this case, which leads to the desired grouping.
This shift/reduce conflict arises only when there is a particular current input symbol, ELSE , and par-
ticular inputs already seen, such as
IF ( C1 ) IF ( C2 ) S1
In general, there may be many conflicts, and each one will be associated with an input symbol and a set of
previously read inputs. The previously read inputs are characterized by the state of the parser.
The conflict messages of Yacc are best understood by examining the verbose (−v) option output file.
For example, the output corresponding to the above conflict state might be:
-- --
state 23
ELSE shift 45
. reduce 18
The first line describes the conflict, giving the state and the input symbol. The ordinary state description
follows, giving the grammar rules active in the state, and the parser actions. Recall that the underline
marks the portion of the grammar rules which has been seen. Thus in the example, in state 23 the parser
has seen input corresponding to
IF ( cond ) stat
and the two grammar rules shown are active at this time. The parser can do two possible things. If the
input symbol is ELSE , it is possible to shift into state 45. State 45 will have, as part of its description, the
line
stat : IF ( cond ) stat ELSEistat
since the ELSE will have been shifted in this state. Back in state 23, the alternative action, described by
‘‘.’’, is to be done if the input symbol is not mentioned explicitly in the above actions; thus, in this case, if
the input symbol is not ELSE , the parser reduces by grammar rule 18:
stat : IF ´(´ cond ´)´ stat
Once again, notice that the numbers following ‘‘shift’’ commands refer to other states, while the numbers
following ‘‘reduce’’ commands refer to grammar rule numbers. In the y.output file, the rule numbers are
printed after those rules which can be reduced. In most one states, there will be at most reduce action pos-
sible in the state, and this will be the default command. The user who encounters unexpected shift/reduce
conflicts will probably want to look at the verbose output to decide whether the default actions are
appropriate. In really tough cases, the user might need to know more about the behavior and construction
of the parser than can be covered here. In this case, one of the theoretical references2-4 might be consulted;
the services of a local guru might also be appropriate.
6: Precedence
There is one common situation where the rules given above for resolving conflicts are not sufficient;
this is in the parsing of arithmetic expressions. Most of the commonly used constructions for arithmetic
expressions can be naturally described by the notion of precedence levels for operators, together with
information about left or right associativity. It turns out that ambiguous grammars with appropriate disam-
biguating rules can be used to create parsers that are faster and easier to write than parsers constructed
from unambiguous grammars. The basic notion is to write grammar rules of the form
expr : expr OP expr
and
expr : UNARY expr
for all binary and unary operators desired. This creates a very ambiguous grammar, with many parsing
conflicts. As disambiguating rules, the user specifies the precedence, or binding strength, of all the opera-
tors, and the associativity of the binary operators. This information is sufficient to allow Yacc to resolve
the parsing conflicts in accordance with these rules, and construct a parser that realizes the desired pre-
cedences and associativities.
-- --
The precedences and associativities are attached to tokens in the declarations section. This is done
by a series of lines beginning with a Yacc keyword: %left, %right, or %nonassoc, followed by a list of
tokens. All of the tokens on the same line are assumed to have the same precedence level and associa-
tivity; the lines are listed in order of increasing precedence or binding strength. Thus,
%left ´+´ ´−´
%left ´∗´ ´/´
describes the precedence and associativity of the four arithmetic operators. Plus and minus are left associa-
tive, and have lower precedence than star and slash, which are also left associative. The keyword %right is
used to describe right associative operators, and the keyword %nonassoc is used to describe operators, like
the operator .LT. in Fortran, that may not associate with themselves; thus,
A .LT. B .LT. C
is illegal in Fortran, and such an operator would be described with the keyword %nonassoc in Yacc. As an
example of the behavior of these declarations, the description
%right ´=´
%left ´+´ ´−´
%left ´∗´ ´/´
%%
%%
A token declared by %left, %right, and %nonassoc need not be, but may be, declared by %token as
well.
The precedences and associativities are used by Yacc to resolve parsing conflicts; they give rise to
disambiguating rules. Formally, the rules work as follows:
1. The precedences and associativities are recorded for those tokens and literals that have them.
2. A precedence and associativity is associated with each grammar rule; it is the precedence and associ-
ativity of the last token or literal in the body of the rule. If the %prec construction is used, it over-
rides this default. Some grammar rules may have no precedence and associativity associated with
them.
3. When there is a reduce/reduce conflict, or there is a shift/reduce conflict and either the input symbol
or the grammar rule has no precedence and associativity, then the two disambiguating rules given at
the beginning of the section are used, and the conflicts are reported.
4. If there is a shift/reduce conflict, and both the grammar rule and the input character have precedence
and associativity associated with them, then the conflict is resolved in favor of the action (shift or
reduce) associated with the higher precedence. If the precedences are the same, then the associa-
tivity is used; left associative implies reduce, right associative implies shift, and nonassociating
implies error.
Conflicts resolved by precedence are not counted in the number of shift/reduce and reduce/reduce
conflicts reported by Yacc. This means that mistakes in the specification of precedences may disguise
errors in the input grammar; it is a good idea to be sparing with precedences, and use them in an essentially
‘‘cookbook’’ fashion, until some experience has been gained. The y.output file is very useful in deciding
whether the parser is actually doing what was intended.
7: Error Handling
Error handling is an extremely difficult area, and many of the problems are semantic ones. When an
error is found, for example, it may be necessary to reclaim parse tree storage, delete or alter symbol table
entries, and, typically, set switches to avoid generating any further output.
It is seldom acceptable to stop all processing when an error is found; it is more useful to continue
scanning the input to find further syntax errors. This leads to the problem of getting the parser ‘‘restarted’’
after an error. A general class of algorithms to do this involves discarding a number of tokens from the
input string, and attempting to adjust the parser so that input can continue.
To allow the user some control over this process, Yacc provides a simple, but reasonably general,
feature. The token name ‘‘error’’ is reserved for error handling. This name can be used in grammar rules;
in effect, it suggests places where errors are expected, and recovery might take place. The parser pops its
stack until it enters a state where the token ‘‘error’’ is legal. It then behaves as if the token ‘‘error’’ were
the current lookahead token, and performs the action encountered. The lookahead token is then reset to the
token that caused the error. If no special error rules have been specified, the processing halts when an error
is detected.
-- --
In order to prevent a cascade of error messages, the parser, after detecting an error, remains in error
state until three tokens have been successfully read and shifted. If an error is detected when the parser is
already in error state, no message is given, and the input token is quietly deleted.
As an example, a rule of the form
stat : error
would, in effect, mean that on a syntax error the parser would attempt to skip over the statement in which
the error was seen. More precisely, the parser will scan ahead, looking for three tokens that might legally
follow a statement, and start processing at the first of these; if the beginnings of statements are not
sufficiently distinctive, it may make a false start in the middle of a statement, and end up reporting a second
error where there is in fact no error.
Actions may be used with these special error rules. These actions might attempt to reinitialize tables,
reclaim symbol table space, etc.
Error rules such as the above are very general, but difficult to control. Somewhat easier are rules
such as
stat : error ´;´
Here, when there is an error, the parser attempts to skip over the statement, but will do so by skipping to
the next ´;´. All tokens after the error and before the next ´;´ cannot be shifted, and are discarded. When
the ´;´ is seen, this rule will be reduced, and any ‘‘cleanup’’ action associated with it performed.
Another form of error rule arises in interactive applications, where it may be desirable to permit a
line to be reentered after an error. A possible error rule might be
input : error ´\n´ { printf( "Reenter last line: " ); } input
{ $$ = $4; }
There is one potential difficulty with this approach; the parser must correctly process three input tokens
before it admits that it has correctly resynchronized after the error. If the reentered line contains an error in
the first two tokens, the parser deletes the offending tokens, and gives no message; this is clearly unaccept-
able. For this reason, there is a mechanism that can be used to force the parser to believe that an error has
been fully recovered from. The statement
yyerrok ;
in an action resets the parser to its normal mode. The last example is better written
input : error ´\n´
{ yyerrok;
printf( "Reenter last line: " ); }
input
{ $$ = $4; }
;
As mentioned above, the token seen immediately after the ‘‘error’’ symbol is the input token at
which the error was discovered. Sometimes, this is inappropriate; for example, an error recovery action
might take upon itself the job of finding the correct place to resume input. In this case, the previous looka-
head token must be cleared. The statement
yyclearin ;
in an action will have this effect. For example, suppose the action after error were to call some sophisti-
cated resynchronization routine, supplied by the user, that attempted to advance the input to the beginning
of the next valid statement. After this routine was called, the next token returned by yylex would presum-
ably be the first token in a legal statement; the old, illegal token must be discarded, and the error state reset.
This could be done by a rule like
-- --
stat : error
{ resynch();
yyerrok ;
yyclearin ; }
;
These mechanisms are admittedly crude, but do allow for a simple, fairly effective recovery of the
parser from many errors; moreover, the user can get control to deal with the error actions required by other
portions of the program.
Input Style
It is difficult to provide rules with substantial actions and still have a readable specification file. The
following style hints owe much to Brian Kernighan.
a. Use all capital letters for token names, all lower case letters for nonterminal names. This rule comes
under the heading of ‘‘knowing who to blame when things go wrong.’’
b. Put grammar rules and actions on separate lines. This allows either to be changed without an
automatic need to change the other.
c. Put all rules with the same left hand side together. Put the left hand side in only once, and let all fol-
lowing rules begin with a vertical bar.
d. Put a semicolon only after the last rule with a given left hand side, and put the semicolon on a
separate line. This allows new rules to be easily added.
e. Indent rule bodies by two tab stops, and action bodies by three tab stops.
The example in Appendix A is written following this style, as are the examples in the text of this
paper (where space permits). The user must make up his own mind about these stylistic questions; the cen-
tral problem, however, is to make the rules visible through the morass of action code.
Left Recursion
The algorithm used by the Yacc parser encourages so called ‘‘left recursive’’ grammar rules: rules of
the form
name : name rest_of_rule ;
These rules frequently arise when writing specifications of sequences and lists:
list : item
| list ´,´ item
;
and
seq : item
| seq item
;
In each of these cases, the first rule will be reduced for the first item only, and the second rule will be
reduced for the second and all succeeding items.
With right recursive rules, such as
seq : item
| item seq
;
the parser would be a bit bigger, and the items would be seen, and reduced, from right to left. More seri-
ously, an internal stack in the parser would be in danger of overflowing if a very long sequence were read.
Thus, the user should use left recursion wherever reasonable.
It is worth considering whether a sequence with zero elements has any meaning, and if so, consider
writing the sequence specification with an empty rule:
seq : /∗ empty ∗/
| seq item
;
Once again, the first rule would always be reduced exactly once, before the first item was read, and then the
second rule would be reduced once for each item read. Permitting empty sequences often leads to
increased generality. However, conflicts might arise if Yacc is asked to decide which empty sequence it
has seen, when it hasn’t seen enough to know!
-- --
Lexical Tie-ins
Some lexical decisions depend on context. For example, the lexical analyzer might want to delete
blanks normally, but not within quoted strings. Or names might be entered into a symbol table in declara-
tions, but not in expressions.
One way of handling this situation is to create a global flag that is examined by the lexical analyzer,
and set by actions. For example, suppose a program consists of 0 or more declarations, followed by 0 or
more statements. Consider:
%{
int dflag;
%}
... other declarations ...
%%
decls : /∗ empty ∗/
{ dflag = 1; }
| decls declaration
;
stats : /∗ empty ∗/
{ dflag = 0; }
| stats statement
;
Reserved Words
Some programming languages permit the user to use words like ‘‘if’’, which are normally reserved,
as label or variable names, provided that such use does not conflict with the legal use of these names in the
programming language. This is extremely hard to do in the framework of Yacc; it is difficult to pass infor-
mation to the lexical analyzer telling it ‘‘this instance of ‘if’ is a keyword, and that instance is a variable’’.
The user can make a stab at it, using the mechanism described in the last subsection, but it is difficult.
A number of ways of making this easier are under advisement. Until then, it is better that the key-
words be reserved ; that is, be forbidden for use as variable names. There are powerful stylistic reasons for
preferring this, anyway.
behave as if the current input symbol had been a syntax error; yyerror is called, and error recovery takes
place. These mechanisms can be used to simulate parsers with multiple endmarkers or context-sensitive
syntax checking.
noun : DOG
{ $$ = DOG; }
| CRONE
{ if( $0 == YOUNG ){
printf( "what?\n" );
}
$$ = CRONE;
}
;
...
In the action following the word CRONE, a check is made that the preceding token shifted was not
YOUNG. Obviously, this is only possible when a great deal is known about what might precede the sym-
bol noun in the input. There is also a distinctly unstructured flavor about this. Nevertheless, at times this
mechanism will save a great deal of trouble, especially when a few combinations are to be excluded from
an otherwise regular structure.
This declares the Yacc value stack, and the external variables yylval and yyval , to have type equal to this
union. If Yacc was invoked with the −d option, the union declaration is copied onto the y.tab.h file. Alter-
natively, the union may be declared in a header file, and a typedef used to define the variable YYSTYPE to
represent this union. Thus, the header file might also have said:
typedef union {
body of union ...
} YYSTYPE;
The header file must be included in the declarations section, by use of %{ and %}.
Once YYSTYPE is defined, the union member names must be associated with the various terminal
and nonterminal names. The construction
< name >
is used to indicate a union member name. If this follows one of the keywords %token, %left, %right, and
%nonassoc, the union member name is associated with the tokens listed. Thus, saying
%left <optype> ´+´ ´−´
will cause any reference to values returned by these two tokens to be tagged with the union member name
optype . Another keyword, %type, is used similarly to associate union member names with nonterminals.
Thus, one might say
%type <nodetype> expr stat
There remain a couple of cases where these mechanisms are insufficient. If there is an action within
a rule, the value returned by this action has no a priori type. Similarly, reference to left context values
(such as $0 − see the previous subsection ) leaves Yacc with no easy way of knowing the type. In this
case, a type can be imposed on the reference by inserting a union member name, between < and >, immedi-
ately after the first $. An example of this usage is
rule : aaa { $<intval>$ = 3; } bbb
{ fun( $<intval>2, $<other>0 ); }
;
This syntax has little to recommend it, but the situation arises rarely.
A sample specification is given in Appendix C. The facilities in this subsection are not triggered
until they are used: in particular, the use of %type will turn on these mechanisms. When they are used,
there is a fairly strict level of checking. For example, use of $n or $$ to refer to something with no defined
type is diagnosed. If these facilities are not triggered, the Yacc value stack is used to hold int’ s, as was
true historically.
11: Acknowledgements
Yacc owes much to a most stimulating collection of users, who have goaded me beyond my inclina-
tion, and frequently beyond my ability, in their endless search for ‘‘one more feature’’. Their irritating
unwillingness to learn how to do things my way has usually led to my doing things their way; most of the
time, they have been right. B. W. Kernighan, P. J. Plauger, S. I. Feldman, C. Imagna, M. E. Lesk, and A.
Snyder will recognize some of their ideas in the current version of Yacc. C. B. Haley contributed to the
error recovery algorithm. D. M. Ritchie, B. W. Kernighan, and M. O. Harris helped translate this docu-
ment into English. Al Aho also deserves special credit for bringing the mountain to Mohammed, and other
favors.
-- --
References
1. B. W. Kernighan and D. M. Ritchie, The C Programming Language, Prentice-Hall, Englewood
Cliffs, New Jersey, 1978.
2. A. V. Aho and S. C. Johnson, ‘‘LR Parsing,’’ Comp. Surveys, vol. 6, no. 2, pp. 99-124, June 1974.
3. A. V. Aho, S. C. Johnson, and J. D. Ullman, ‘‘Deterministic Parsing of Ambiguous Grammars,’’
Comm. Assoc. Comp. Mach., vol. 18, no. 8, pp. 441-452, August 1975.
4. A. V. Aho and J. D. Ullman, Principles of Compiler Design, Addison-Wesley, Reading, Mass., 1977.
5. S. C. Johnson, ‘‘Lint, a C Program Checker,’’ Comp. Sci. Tech. Rep. No. 65, 1978. updated version
TM 78-1273-3
6. S. C. Johnson, ‘‘A Portable Compiler: Theory and Practice,’’ Proc. 5th ACM Symp. on Principles of
Programming Languages, pp. 97-104, January 1978.
7. B. W. Kernighan and L. L. Cherry, ‘‘A System for Typesetting Mathematics,’’ Comm. Assoc. Comp.
Mach., vol. 18, pp. 151-157, Bell Laboratories, Murray Hill, New Jersey, March 1975.
8. M. E. Lesk, ‘‘Lex — A Lexical Analyzer Generator,’’ Comp. Sci. Tech. Rep. No. 39, Bell Labora-
tories, Murray Hill, New Jersey, October 1975.
-- --
%{
# include <stdio.h>
# include <ctype.h>
int regs[26];
int base;
%}
%start list
%left ´|´
%left ´&´
%left ´+´ ´−´
%left ´∗´ ´/´ ´%´
%left UMINUS /∗ supplies precedence for unary minus ∗/
list : /∗ empty ∗/
| list stat ´\n´
| list error ´\n´
{ yyerrok; }
;
stat : expr
{ printf( "%d\n", $1 ); }
| LETTER ´=´ expr
{ regs[$1] = $3; }
;
number : DIGIT
{ $$ = $1; base = ($1==0) ? 8 : 10; }
| number DIGIT
{ $$ = base ∗ $1 + $2; }
;
%% /∗ start of programs ∗/
int c;
/∗ c is now nonblank ∗/
if( islower( c ) ) {
yylval = c − ´a´;
return ( LETTER );
}
if( isdigit( c ) ) {
yylval = c − ´0´;
return( DIGIT );
}
return( c );
}
-- --
/∗ basic entities ∗/
%token IDENTIFIER /∗ includes identifiers and literals ∗/
%token C_IDENTIFIER /∗ identifier (but not literal) followed by colon ∗/
%token NUMBER /∗ [0-9]+ ∗/
%start spec
%%
defs : /∗ empty ∗/
| defs def
;
rword : TOKEN
| LEFT
| RIGHT
| NONASSOC
-- --
| TYPE
;
nlist : nmno
| nlist nmno
| nlist ´,´ nmno
;
/∗ rules section ∗/
rbody : /∗ empty ∗/
| rbody IDENTIFIER
| rbody act
;
prec : /∗ empty ∗/
| PREC IDENTIFIER
| PREC IDENTIFIER act
| prec ´;´
;
-- --
%{
# include <stdio.h>
# include <ctype.h>
double atof();
double dreg[ 26 ];
INTERVAL vreg[ 26 ];
%}
%start lines
%union {
int ival;
double dval;
INTERVAL vval;
}
%%
lines : /∗ empty ∗/
| lines line
;
{ vreg[$1] = $3; }
| error ´\n´
{ yyerrok; }
;
dexp : CONST
| DREG
{ $$ = dreg[$1]; }
| dexp ´+´ dexp
{ $$ = $1 + $3; }
| dexp ´−´ dexp
{ $$ = $1 − $3; }
| dexp ´∗´ dexp
{ $$ = $1 ∗ $3; }
| dexp ´/´ dexp
{ $$ = $1 / $3; }
| ´−´ dexp %prec UMINUS
{ $$ = − $2; }
| ´(´ dexp ´)´
{ $$ = $2; }
;
vexp : dexp
{ $$.hi = $$.lo = $1; }
| ´(´ dexp ´,´ dexp ´)´
{
$$.lo = $2;
$$.hi = $4;
if( $$.lo > $$.hi ){
printf( "interval out of order\n" );
YYERROR;
}
}
| VREG
{ $$ = vreg[$1]; }
| vexp ´+´ vexp
{ $$.hi = $1.hi + $3.hi;
$$.lo = $1.lo + $3.lo; }
| dexp ´+´ vexp
{ $$.hi = $1 + $3.hi;
$$.lo = $1 + $3.lo; }
| vexp ´−´ vexp
{ $$.hi = $1.hi − $3.lo;
$$.lo = $1.lo − $3.hi; }
| dexp ´−´ vexp
{ $$.hi = $1 − $3.lo;
$$.lo = $1 − $3.hi; }
| vexp ´∗´ vexp
{ $$ = vmul( $1.lo, $1.hi, $3 ); }
| dexp ´∗´ vexp
{ $$ = vmul( $1, $1, $3 ); }
| vexp ´/´ vexp
{ if( dcheck( $3 ) ) YYERROR;
$$ = vdiv( $1.lo, $1.hi, $3 ); }
-- --
%%
/∗ lexical analysis ∗/
yylex(){
register c;
if( isupper( c ) ){
yylval.ival = c − ´A´;
return( VREG );
}
if( islower( c ) ){
yylval.ival = c − ´a´;
return( DREG );
}
∗cp = c;
if( isdigit( c ) ) continue;
if( c == ´.´ ){
if( dot++ || exp ) return( ´.´ ); /∗ will cause syntax error ∗/
continue;
}
if( c == ´e´ ){
if( exp++ ) return( ´e´ ); /∗ will cause syntax error ∗/
continue;
}
/∗ end of number ∗/
break;
}
∗cp = ´\0´;
if( (cp−buf) >= BSZ ) printf( "constant too long: truncated\n" );
-- --
if( c>d ) {
if( c>v.hi ) v.hi = c;
if( d<v.lo ) v.lo = d;
}
else {
if( d>v.hi ) v.hi = d;
if( c<v.lo ) v.lo = c;
}
return( v );
}
dcheck( v ) INTERVAL v; {
if( v.hi >= 0. && v.lo <= 0. ){
printf( "divisor interval contains 0.\n" );
return( 1 );
}
return( 0 );
}