5/11/2016

Bison1.25HandlingContextDependencies

Gotothefirst,previous,next,lastsection,tableofcontents.

HandlingContextDependencies
TheBisonparadigmistoparsetokensfirst,thengroupthemintolargersyntacticunits.Inmany
languages,themeaningofatokenisaffectedbyitscontext.AlthoughthisviolatestheBisonparadigm,
certaintechniques(knownaskludges)mayenableyoutowriteBisonparsersforsuchlanguages.
(Actually,"kludge"meansanytechniquethatgetsitsjobdonebutisneithercleannorrobust.)

SemanticInfoinTokenTypes
TheClanguagehasacontextdependency:thewayanidentifierisuseddependsonwhatitscurrent
meaningis.Forexample,considerthis:
foo(x);

Thislookslikeafunctioncallstatement,butiffooisatypedefname,thenthisisactuallyadeclarationof
x.HowcanaBisonparserforCdecidehowtoparsethisinput?
ThemethodusedinGNUCistohavetwodifferenttokentypes,IDENTIFIERandTYPENAME.Whenyylex
findsanidentifier,itlooksupthecurrentdeclarationoftheidentifierinordertodecidewhichtokentype
toreturn:TYPENAMEiftheidentifierisdeclaredasatypedef,IDENTIFIERotherwise.
Thegrammarrulescanthenexpressthecontextdependencybythechoiceoftokentypetorecognize.
IDENTIFIERisacceptedasanexpression,butTYPENAMEisnot.TYPENAMEcanstartadeclaration,but
IDENTIFIERcannot.Incontextswherethemeaningoftheidentifierisnotsignificant,suchasin
declarationsthatcanshadowatypedefname,eitherTYPENAMEorIDENTIFIERisacceptedthereisonerule
foreachofthetwotokentypes.
Thistechniqueissimpletouseifthedecisionofwhichkindsofidentifierstoallowismadeataplace
closetowheretheidentifierisparsed.ButinCthisisnotalwaysso:Callowsadeclarationtoredeclarea
typedefnameprovidedanexplicittypehasbeenspecifiedearlier:
typedefintfoo,bar,lose;
staticfoo(bar);/*redeclarebarasstaticvariable*/
staticintfoo(lose);/*redeclarefooasfunction*/

Unfortunately,thenamebeingdeclaredisseparatedfromthedeclarationconstructitselfbya
complicatedsyntacticstructurethe"declarator".
Asaresult,thepartofBisonparserforCneedstobeduplicated,withallthenonterminalnames
changed:onceforparsingadeclarationinwhichatypedefnamecanberedefined,andonceforparsinga
declarationinwhichthatcan'tbedone.Hereisapartoftheduplication,withactionsomittedforbrevity:
initdcl:
declaratormaybeasm'='
init
|declaratormaybeasm
;
http://dinosaur.compilertools.net/bison/bison_10.html#SEC82

1/3

5/11/2016

Bison1.25HandlingContextDependencies

notype_initdcl:
notype_declaratormaybeasm'='
init
|notype_declaratormaybeasm
;

Hereinitdclcanredeclareatypedefname,butnotype_initdclcannot.Thedistinctionbetween
declaratorandnotype_declaratoristhesamesortofthing.
Thereissomesimilaritybetweenthistechniqueandalexicaltiein(describednext),inthatinformation
whichaltersthelexicalanalysisischangedduringparsingbyotherpartsoftheprogram.Thedifference
isheretheinformationisglobal,andisusedforotherpurposesintheprogram.Atruelexicaltieinhasa
specialpurposeflagcontrolledbythesyntacticcontext.

LexicalTieins
Onewaytohandlecontextdependencyisthelexicaltiein:aflagwhichissetbyBisonactions,whose
purposeistoalterthewaytokensareparsed.
Forexample,supposewehavealanguagevaguelylikeC,butwithaspecialconstruct`hex(hexexpr)'.
Afterthekeywordhexcomesanexpressioninparenthesesinwhichallintegersarehexadecimal.In
particular,thetoken`a1b'mustbetreatedasanintegerratherthanasanidentifierifitappearsinthat
context.Hereishowyoucandoit:
%{
inthexflag;
%}
%%
...
expr:IDENTIFIER
|constant
|HEX'('
{hexflag=1;}
expr')'
{hexflag=0;
$$=$4;}
|expr'+'expr
{$$=make_sum($1,$3);}
...
;
constant:
INTEGER
|STRING
;

Hereweassumethatyylexlooksatthevalueofhexflagwhenitisnonzero,allintegersareparsedin
hexadecimal,andtokensstartingwithlettersareparsedasintegersifpossible.
ThedeclarationofhexflagshownintheCdeclarationssectionoftheparserfileisneededtomakeit
accessibletotheactions(seesectionTheCDeclarationsSection).Youmustalsowritethecodeinyylex
toobeytheflag.

http://dinosaur.compilertools.net/bison/bison_10.html#SEC82

2/3

5/11/2016

Bison1.25HandlingContextDependencies

LexicalTieinsandErrorRecovery
Lexicaltieinsmakestrictdemandsonanyerrorrecoveryrulesyouhave.SeesectionErrorRecovery.
Thereasonforthisisthatthepurposeofanerrorrecoveryruleistoaborttheparsingofoneconstruct
andresumeinsomelargerconstruct.Forexample,inClikelanguages,atypicalerrorrecoveryruleisto
skiptokensuntilthenextsemicolon,andthenstartanewstatement,likethis:
stmt:expr';'
|IF'('expr')'stmt{...}
...
error';'
{hexflag=0;}
;

Ifthereisasyntaxerrorinthemiddleofa`hex(expr)'construct,thiserrorrulewillapply,andthenthe
actionforthecompleted`hex(expr)'willneverrun.Sohexflagwouldremainsetfortheentirerestof
theinput,oruntilthenexthexkeyword,causingidentifierstobemisinterpretedasintegers.
Toavoidthisproblemtheerrorrecoveryruleitselfclearshexflag.
Theremayalsobeanerrorrecoveryrulethatworkswithinexpressions.Forexample,therecouldbea
rulewhichapplieswithinparenthesesandskipstothecloseparenthesis:
expr:...
|'('expr')'
{$$=$2;}
|'('error')'
...

Ifthisruleactswithinthehexconstruct,itisnotgoingtoabortthatconstruct(sinceitappliestoaninner
levelofparentheseswithintheconstruct).Therefore,itshouldnotcleartheflag:therestofthehex
constructshouldbeparsedwiththeflagstillineffect.
Whatifthereisanerrorrecoveryrulewhichmightabortoutofthehexconstructormightnot,depending
oncircumstances?Thereisnowayyoucanwritetheactiontodeterminewhetherahexconstructisbeing
abortedornot.Soifyouareusingalexicaltiein,youhadbettermakesureyourerrorrecoveryrulesare
notofthiskind.Eachrulemustbesuchthatyoucanbesurethatitalwayswill,oralwayswon't,haveto
cleartheflag.
Gotothefirst,previous,next,lastsection,tableofcontents.

http://dinosaur.compilertools.net/bison/bison_10.html#SEC82

3/3