docs.sun.com · Programming Utilities Guide
docs.sun.com Home Collection Table of Contents Book Table of Contents Print Help
Search this Book Tips


yacc -- A Compiler Compiler

3

yacc (yet another compiler compiler) provides a general tool for imposing structure on the input to a computer program. Before using yacc, you prepare a specification that includes:
yacc then turns the specification into a C-language function that examines the input stream. This function, called a parser, works by calling the low-level scanner.
The scanner, called a lexical analyzer, picks up items from the input stream. The selected items are known as tokens. Tokens are compared to the input construct rules, called grammar rules.
When one of the rules is recognized, the code you have supplied for the rule is invoked. This code is called an action. Actions are fragments of C-language code. They can return values and use values returned by other actions.
The heart of the yacc specification is the collection of grammar rules. Each rule describes a construct and gives it a name. For example, one grammar rule might be:

  date: month_name day ',' year ;
where date, month_name, day, and year represent constructs of interest; presumably, month_name, day, and year are defined in greater detail elsewhere.
In the example, the comma is enclosed in single quotes. This means that the comma is to appear literally in the input. The colon and semicolon are punctuation in the rule and have no significance in evaluating the input. With proper definitions, the input:

  July 4, 1776
might be matched by the rule.
The lexical analyzer is an important part of the parsing function. This user-supplied routine reads the input stream, recognizes the lower-level constructs, and communicates these as tokens to the parser. The lexical analyzer recognizes constructs of the input stream as terminal symbols; the parser recognizes constructs as nonterminal symbols. To avoid confusion, refer to terminal symbols as tokens.
There is considerable leeway in deciding whether to recognize constructs using the lexical analyzer or grammar rules. For example, the rules:

  month_name : 'J' 'a' 'n' ;  
  month_name : 'F' 'e' 'b' ;  
  ...  
  month_name : 'D' 'e' 'c' ;
might be used in the above example. While the lexical analyzer only needs to recognize individual letters, such low-level rules tend to waste time and space and may complicate the specification beyond the ability of yacc to deal with it.
Usually, the lexical analyzer recognizes the month names and returns an indication that a month_name is seen. In this case, month_name is a token and the detailed rules are not needed.
Literal characters such as a comma must also be passed through the lexical analyzer and are also considered tokens.
Specification files are very flexible. It is relatively easy to add to the previous example the rule:

  date : month '/' day '/' year ;
allowing:

  7/4/1776
as a synonym for:

  July 4, 1776
on input. In most cases, this new rule could be slipped into a working system with minimal effort and little danger of disrupting existing input.
The input being read may not conform to the specifications. With a left-to-right scan, input errors are detected as early as is theoretically possible. Thus, not only is the chance of reading and computing with bad input data substantially reduced, but the bad data usually can be found quickly.
Error handling, provided as part of the input specifications, permits the reentry of bad data or the continuation of the input process after skipping over the bad data. In some cases, yacc fails to produce a parser when given a set of specifications.
For example, the specifications may be self-contradictory, or they may require a more powerful recognition mechanism than that available to yacc.
The former cases represent design errors; the latter cases often can be corrected by making the lexical analyzer more powerful or by rewriting some of the grammar rules.
While yacc cannot handle all possible specifications, its power compares favorably with similar systems. Moreover, the constructs that are difficult for yacc to handle are also frequently difficult for human beings to handle. Some users have reported that the discipline of formulating valid yacc specifications for their input revealed errors of conception or design early in program development.
The remainder of this chapter describes the following subjects:
In addition, there are two examples and a summary of the yacc input syntax.

Internationalization

To use yacc in the development of applications in languages other than English, see yacc(1) for further information.

Basic Specifications

Names refer to either tokens or nonterminal symbols. yacc requires token names to be declared as such. While the lexical analyzer may be included as part of the specification file, it is perhaps more in keeping with modular design to keep it as a separate file. Like the lexical analyzer, other subroutines may be included as well.
Thus, every specification file theoretically consists of three sections: the declarations, (grammar) rules, and subroutines. The sections are separated by double percent signs (%%; the percent sign is generally used in yacc specifications as an escape character).
When all sections are used, a full specification file looks like:

  declarations  
  %%  
  rules  
  %%  
  subroutines
The declarations and subroutines sections are optional. The smallest legal yacc specification might be:

  %%  
  S:;
Blanks, tabs, and newlines are ignored, but they may not appear in names or multicharacter reserved symbols. Comments may appear wherever a name is legal. They are enclosed in /* and */, as in the C language.
The rules section is made up of one or more grammar rules. A grammar rule has the form:

  A: BODY ;
where A represents a nonterminal symbol, and BODY represents a sequence of zero or more names and literals. The colon and the semicolon are yacc punctuation.
Names may be of any length and may be made up of letters, periods, underscores, and digits although a digit may not be the first character of a name. Uppercase and lowercase letters are distinct. The names used in the body of a grammar rule may represent tokens or nonterminal symbols.
A literal consists of a character enclosed in single quotes. As in the C language, the backslash is an escape character within literals. yacc recognizes all C language escape sequences. For a number of technical reasons, the null character should never be used in grammar rules.
If there are several grammar rules with the same left-hand side, the vertical bar can be used to avoid rewriting the left-hand side. In addition, the semicolon at the end of a rule is dropped before a vertical bar.
Thus the grammar rules:

  A : B C D ;  
  A : E F ;  
  A : G ;
can be given to yacc as:

  A   : B C D  
       | E F  
       | G  
       ;
by using the vertical bar. It is not necessary that all grammar rules with the same left side appear together in the grammar rules section although it makes the input more readable and easier to change.
If a nonterminal symbol matches the empty string, this can be indicated by:

  epsilon : ;
The blank space following the colon is understood by yacc to be a nonterminal symbol named epsilon.
Names representing tokens must be declared. This is most simply done by writing:

  $token name1 name2 name3
and so on in the declarations section. Every name not defined in the declarations section is assumed to represent a nonterminal symbol. Every nonterminal symbol must appear on the left side of at least one rule.
Of all the nonterminal symbols, the start symbol has particular importance. By default, the symbol is taken to be the left-hand side of the first grammar rule in the rules section. It is possible and desirable to declare the start symbol explicitly in the declarations section using the %start keyword:

  $start symbol
The end of the input to the parser is signaled by a special token, called the end-marker. The end-marker is represented by either a zero or a negative number.
If the tokens up to but not including the end-marker form a construct that matches the start symbol, the parser function returns to its caller after the end-marker is seen and accepts the input. If the end-marker is seen in any other context, it is an error.
It is the job of the user-supplied lexical analyzer to return the end-marker when appropriate. Usually the end-marker represents some reasonably obvious I/O status, such as end of file or end of record.

Actions

With each grammar rule, you can associate actions to be performed when the rule is recognized. Actions may return values and may obtain the values returned by previous actions. Moreover, the lexical analyzer can return values for tokens if desired.
An action is an arbitrary C-language statement and as such can do input and output, call subroutines, and alter arrays and variables. An action is specified by one or more statements enclosed in { and }. For example, the following two examples are grammar rules with actions:

  A   : '(' B ')'  
       {  
           hello( 1, "abc" );  
       }
and

  XXX : YYY ZZZ  
       {  
           (void) printf("a message\n");  
           flag = 25;  
       }
The $ symbol is used to facilitate communication between the actions and the parser. The pseudo-variable $$ represents the value returned by the complete action.
For example, the action:

  { $$ = 1; }
returns the value of one; in fact, that's all it does.
To obtain the values returned by previous actions and the lexical analyzer, the action can use the pseudo-variables $1, $2, ... $n. These refer to the values returned by components 1 through n of the right side of a rule, with the components being numbered from left to right. If the rule is

  A   : B C D ;
then $2 has the value returned by C, and $3 the value returned by D. The following rule provides a common example:

  expr    : '(' expr ')' ;
One would expect the value returned by this rule to be the value of the expr within the parentheses. Since the first component of the action is the literal left parenthesis, the desired logical result can be indicated by:

  expr    : '(' expr ')'  
           {  
               $$ = $2 ;  
           }
By default, the value of a rule is the value of the first element in it ($1). Thus, grammar rules of the following form frequently need not have an explicit action:

  A : B ;
In previous examples, all the actions came at the end of rules. Sometimes, it is desirable to get control before a rule is fully parsed. yacc permits an action to be written in the middle of a rule as well as at the end.
This action is assumed to return a value accessible through the usual $ mechanism by the actions to the right of it. In turn, it may access the values returned by the symbols to its left. Thus, in the rule below the effect is to set x to 1 and y to the value returned by C:

  A   : B  
           {  
               $$ = 1;  
           }  
           C  
       {  
           x = $2;  
           y = $3;  
       }  
       ;
Actions that do not terminate a rule are handled by yacc by manufacturing a new nonterminal symbol name and a new rule matching this name to the empty string. The interior action is the action triggered by recognizing this added rule.
yacc treats the above example as if it had been written

  $ACT    :   /* empty */  
           {  
               $$ = 1;  
           }  
           ;  
  A       :   B $ACT C  
           {  
               x = $2;  
               y = $3;  
           }  
           ;
where $ACT is an empty action.
In many applications, output is not done directly by the actions. A data structure, such as a parse tree, is constructed in memory and transformations are applied to it before output is generated. Parse trees are particularly easy to construct given routines to build and maintain the tree structure desired.
For example, suppose there is a C-function node written so that the call:

  node( L, n1, n2 )
creates a node with label L and descendants n1 and n2 and returns the index of the newly created node. Then a parse tree can be built by supplying actions such as in the following specification:

  expr    : expr '+' expr  
           {  
           $$ = node( '+', $1, $3 );  
           }
You may define other variables to be used by the actions. Declarations and definitions can appear in the declarations section enclosed in %{ and %}. These declarations and definitions have global scope, so they are known to the action statements and can be made known to the lexical analyzer. For example:

  %{ int variable = 0; %}
could be placed in the declarations section making variable accessible to all of the actions. You should avoid names beginning with yy because the yacc parser uses only such names. Note, too, that in the examples shown thus far all the values are integers.
A discussion of values is found in the section Advanced Topics. Finally, note that in the following case:

  %{  
       int i;  
       printf("%}");  
  %}
yacc will start copying after %{ and stop copying when it encounters the first %}, the one in printf(). In contrast, it would copy %{ in printf() if it encountered it there.

Lexical Analysis

You must supply a lexical analyzer to read the input stream and communicate tokens (with values, if desired) to the parser. The lexical analyzer is an integer-valued function called yylex(). The function returns an integer, the token number, representing the kind of token read. If a value is associated with that token, it should be assigned to the external variable yylval.
The parser and the lexical analyzer must agree on these token numbers in order for communication between them to take place. The numbers may be chosen by yacc or the user. In either case, the #define mechanism of C language is used to allow the lexical analyzer to return these numbers symbolically.
For example, suppose that the token name DIGIT has been defined in the declarations section of the yacc specification file. The relevant portion of the lexical analyzer might look like the following to return the appropriate token:

  int yylex()  
  {  
       extern int yylval;  
       int c;  
       ...  
       c = getchar();  
       ...  
       switch (c)  
       {  
           ...  
       case '0':  
       case '1':  
       ...  
       case '9':  
           yylval = c - '0';  
           return (DIGIT);  
       ...  
       }  
       ...  
  }
The intent is to return a token number of DIGIT and a value equal to the numerical value of the digit. You put the lexical analyzer code in the subroutines section and the declaration for DIGIT in the declarations section. Alternatively, you can put the lexical analyzer code in a separately compiled file, provided you
This mechanism leads to clear, easily modified lexical analyzers. The only pitfall to avoid is the use of any token names in the grammar that are reserved or significant in C language or the parser.
For example, the use of token names if or while will almost certainly cause severe difficulties when the lexical analyzer is compiled. The token name error is reserved for error handling and should not be used naively.
In the default situation, token numbers are chosen by yacc. The default token number for a literal character is the numerical value of the character in the local character set. Other names are assigned token numbers starting at 257.
If you prefer to assign the token numbers, the first appearance of the token name or literal in the declarations section must be followed immediately by a nonnegative integer. This integer is taken to be the token number of the name or literal. Names and literals not defined this way are assigned default definitions by yacc. The potential for duplication exists here. Care must be taken to make sure that all token numbers are distinct.
For historical reasons, the end-marker must have token number 0 or negative. You cannot redefine this token number. Thus, all lexical analyzers should be prepared to return 0 or a negative number as a token upon reaching the end of their input.
As noted in Chapter 2, lex-Lexical Analysis, lexical analyzers produced by lex are designed to work in close harmony with yacc parsers. The specifications for these lexical analyzers use regular expressions instead of grammar rules. lex can be used to produce quite complicated lexical analyzers, but there remain some languages that do not fit any theoretical framework and whose lexical analyzers must be crafted by hand.

Parser Operation

Use yacc to turn the specification file into a C language procedure, which parses the input according to the specification given. The algorithm used to go from the specification to the parser is complex and will not be discussed here. The parser itself, though, is relatively simple and understanding its usage will make treatment of error recovery and ambiguities easier.
The parser produced by yacc consists of a finite state machine with a stack. The parser is also capable of reading and remembering the next input token, called the lookahead token. The current state is always the one on the top of the stack. The states of the finite state machine are given small integer labels. Initially, the machine is in state 0 (the stack contains only state 0) and no lookahead token has been read.
The machine has only four actions available: shift, reduce, accept, and error. A step of the parser is done as follows:
  1. Based on its current state, the parser decides if it needs a look-ahead token to choose the action to be taken. If it needs one and does not have one, it calls yylex() to obtain the next token.

  2. Using the current state and the lookahead token if needed, the parser decides on its next action and carries it out. This may result in states being pushed onto the stack or popped off of the stack and in the lookahead token being processed or left alone.

The shift action is the most common action the parser takes. Whenever a shift action is taken, there is always a lookahead token. For example, in state 56 there may be an action

  IF shift 34
that says, in state 56, if the lookahead token is IF, the current state (56) is pushed down on the stack, and state 34 becomes the current state (on the top of the stack). The lookahead token is cleared.
The reduce action keeps the stack from growing without bounds. reduce actions are appropriate when the parser has seen the right-hand side of a grammar rule and is prepared to announce that it has seen an instance of the rule replacing the right-hand side by the left-hand side. It may be necessary to consult the lookahead token to decide whether or not to reduce. In fact, the default action (represented by .) is often a reduce action.
reduce actions are associated with individual grammar rules. Grammar rules are also given small integer numbers, and this leads to some confusion. The following action refers to grammar rule 18:

  .  reduce 18
However, the following action refers to state 34:

  IF shift 34
Suppose the following rule is being reduced:

  A : x y z ;
The reduce action depends on the left-hand symbol (A in this case) and the number of symbols on the right-hand side (three in this case). To reduce, first pop off the top three states from the stack. (In general, the number of states popped equals the number of symbols on the right side of the rule.)
In effect, these states were the ones put on the stack while recognizing x, y, and z and no longer serve any useful purpose. After popping these states, a state is uncovered, which was the state the parser was in before beginning to process the rule.
Using this uncovered state and the symbol on the left side of the rule, perform what is in effect a shift of A. A new state is obtained, pushed onto the stack, and parsing continues. There are significant differences between the processing of the left-hand symbol and an ordinary shift of a token, however, so this action is called a goto action. In particular, the lookahead token is cleared by a shift but is not affected by a goto. In any case, the uncovered state contains an entry such as the following causing state 20 to be pushed onto the stack and become the current state:

  A goto 20
In effect, the reduce action turns back the clock in the parse, popping the states off the stack to go back to the state where the right-hand side of the rule was first seen. The parser then behaves as if it had seen the left side at that time. If the right-hand side of the rule is empty, no states are popped off the stacks. The uncovered state is in fact the current state.
The reduce action is also important in the treatment of user-supplied actions and values. When a rule is reduced, the code supplied with the rule is executed before the stack is adjusted. In addition to the stack holding the states, another stack running in parallel with it holds the values returned from the lexical analyzer and the actions.
When a shift takes place, the external variable yylval is copied onto the value stack. After the return from the user code, the reduction is carried out. When the goto action is done, the external variable yyval is copied onto the value stack. The pseudo-variables $1, $2, and so on refer to the value stack.
The other two parser actions are conceptually much simpler. The accept action indicates that the entire input has been seen and that it matches the specification. This action appears only when the lookahead token is the end-marker and indicates that the parser has successfully done its job.
The error action, on the other hand, represents a place where the parser can no longer continue parsing according to the specification. The input tokens it has seen (together with the lookahead token) cannot be followed by anything that would result in a legal input. The parser reports an error and attempts to recover the situation and resume parsing.
The error recovery (as opposed to the detection of error) is be discussed in the Error Handling section.
Consider the following as a yacc specification:

  $token DING DONG DELL  
  %%  
  rhyme   : sound place  
           ;  
  sound   : DING DONG  
           ;  
  place: DELL  
           ;
When yacc is invoked with the -v (verbose) option, a file called y.output is produced which describes the parser.
The y.output file corresponding to the above grammar (with some statistics stripped off the end) follows.

  state 0  


           $accept : _rhyme $end  
           DING shift 3  
           .  error  
           rhyme goto 1  
           sound goto 2  
  state 1  
           $accept : rhyme_$end  
           $end accept  
           .  error  
  state 2  
           rhyme : sound_place  
           DELL shift 5  
           .  error  
           place   goto 4  
  state 3  
           sound : DING_DONG  
           DONG shift 6  
           .  error  


  state 4  
           rhyme : sound place_ (1)  
           . reduce 1  
  state 5  
           place : DELL_ (3)  
           . reduce 3  
  state 6  
           sound : DING DONG_ (2)  
           .  reduce 2
The actions for each state are specified and there is a description of the parsing rules being processed in each state. The _ character is used to indicate what has been seen and what is yet to come in each rule. The following input can be used to track the operations of the parser:

  DING DONG DELL
Initially, the current state is state 0.
The parser refers to the input to decide between the actions available in state 0, so the first token, DING, is read and becomes the lookahead token. The action in state 0 on DING is shift 3, state 3 is pushed onto the stack, and the lookahead token is cleared. State 3 becomes the current state. The next token, DONG, is read and becomes the lookahead token.
The action in state 3 on the token DONG is shift 6, state 6 is pushed onto the stack, and the lookahead is cleared. The stack now contains 0, 3, and 6. In state 6, without consulting the lookahead, the parser reduces by

  sound : DING DONG
which is rule 2. Two states, 6 and 3, are popped off the stack, uncovering state 0. Consulting the description of state 0 (looking for a goto on sound),

  sound goto 2
is obtained. State 2 is pushed onto the stack and becomes the current state.
In state 2, the next token, DELL, must be read. The action is shift 5, so state 5 is pushed onto the stack, which now has 0, 2, and 5 on it, and the lookahead token is cleared. In state 5, the only action is to reduce by rule 3. This has one symbol on the right-hand side, so one state, 5, is popped off, and state 2 is uncovered.
The goto in state 2 on place (the left side of rule 3) is state 4. Now, the stack contains 0, 2, and 4. In state 4, the only action is to reduce by rule 1. There are two symbols on the right, so the top two states are popped off, uncovering state 0 again.

Ambiguity and Conflicts

A set of grammar rules is ambiguous if some input string can be structured in two or more different ways. For example, the following grammar rule is a natural way of expressing the fact that one way of forming an arithmetic expression is to put two other expressions together with a minus sign between them:

  expr : expr '-' expr
Notice that this grammar rule does not completely specify the way that all complex inputs should be structured. For example, if the input is the following:

  expr - expr - expr
the rule allows this input to be structured as either:

  ( expr - expr ) - expr
or as:

  expr - ( expr - expr )
The first is called left association, the second right association.
yacc detects such ambiguities when it is attempting to build the parser. Given that the input is as follows, consider the problem that confronts the parser:

  expr - expr - expr
When the parser has read the second expr, the input seen is:

  expr - expr
It matches the right side of the grammar rule above. The parser could reduce the input by applying this rule. After applying the rule, the input is reduced to expr (the left side of the rule). The parser would then read the final part of the input (as represented below) and again reduce:

  - expr
The effect of this is to take the left associative interpretation.
Alternatively, if the parser sees the following:

  expr - expr
it could defer the immediate application of the rule and continue reading the input until the following is seen:

  expr - expr - expr
It could then apply the rule to the rightmost three symbols, reducing them to expr, which results in the following being left:

  expr - expr
Now the rule can be reduced once more. The effect is to take the right associative interpretation. Thus, having read the following, the parser can do one of two legal things, shift or reduce:

  expr - expr
It has no way of deciding between them. This is called a shift-reduce conflict. It may also happen that the parser has a choice of two legal reductions. This is called a reduce-reduce conflict. Note that there are never any shift-shift conflicts.
When there are shift-reduce or reduce-reduce conflicts, yacc still produces a parser. It does this by selecting one of the valid steps wherever it has a choice. A rule describing the choice to make in a given situation is called a disambiguating rule.
yacc invokes two default disambiguating rules:
  1. In a shift-reduce conflict, the default is to shift.

  2. In a reduce-reduce conflict, the default is to reduce by the earlier grammar rule (in the yacc specification).

Rule 1 implies that reductions are deferred in favor of shifts when there is a choice. Rule 2 gives the user rather crude control over the behavior of the parser in this situation, but reduce-reduce conflicts should be avoided when possible.
Conflicts may arise because of mistakes in input or logic or because the grammar rules (while consistent) require a more complex parser than yacc can construct. The use of actions within rules can also cause conflicts if the action must be done before the parser can be sure which rule is being recognized.
In these cases, the application of disambiguating rules is inappropriate and leads to an incorrect parser. For this reason, yacc always reports the number of shift-reduce and reduce-reduce conflicts resolved by rules 1 and 2 above.
In general, whenever it is possible to apply disambiguating rules to produce a correct parser, it is also possible to rewrite the grammar rules so that the same inputs are read but there are no conflicts. For this reason, most previous parser generators have considered conflicts to be fatal errors.
This rewriting is somewhat unnatural and produces slower parsers. Thus, yacc will produce parsers even in the presence of conflicts.
As an example of the power of disambiguating rules, consider:

  stat    : IF '(' cond ')' stat  
           | IF '(' cond ')' stat ELSE stat  
           ;
which is a fragment from a programming language involving an if-then-else statement. In these rules, IF and ELSE are tokens, cond is a nonterminal symbol describing conditional (logical) expressions, and stat is a nonterminal symbol describing statements. The first rule will be called the simple if rule and the second the if-else rule.
These two rules form an ambiguous construction because input of the form:

  IF ( C1 ) IF ( C2 ) S1 ELSE S2
can be structured according to these rules in two ways:

  IF ( C1 )  
  {  
       IF ( C2 )  
           S1  
  }  
  ELSE  
       S2
or:

  IF ( C1 )  
  {  
       IF ( C2 )  
           S1  
       ELSE  
           S2  
  }
where the second interpretation is the one given in most programming languages having this construct; each ELSE is associated with the last preceding un-ELSE'd IF. In this example, consider the situation where the parser has seen the following and is looking at the ELSE:

  IF ( C1 ) IF ( C2 ) S1
It can immediately reduce by the simple if rule to get:

  IF ( C1 ) stat
and then read the remaining input:

  ELSE S2
and reduce:

  IF ( C1 ) stat ELSE S2
by the if-else rule. This leads to the first of the above groupings of the input.
On the other hand, the ELSE may be shifted, S2 read, and then the right-hand portion of:

  IF ( C1 ) IF ( C2 ) S1 ELSE S2
can be reduced by the if-else rule to get:

  IF ( C1 ) stat
which can be reduced by the simple if rule.
This leads to the second of the above groupings of the input, which is usually the one desired.
Once again, the parser can do two valid things -- there is a shift-reduce conflict. The application of disambiguating rule 1 tells the parser to shift in this case, which leads to the desired grouping.
This shift-reduce conflict arises only when there is a particular current input symbol, ELSE, and particular inputs, such as:

  IF ( C1 ) IF ( C2 ) S1
have already been seen. In general, there may be many conflicts, and each one will be associated with an input symbol and a set of previously read inputs. The previously read inputs are characterized by the state of the parser.
The conflict messages of yacc are best understood by examining the -v output. For example, the output corresponding to the above conflict state might be:

  23: shift-reduce conflict (shift 45, reduce 18) on ELSE  
  
  state 23  
       stat : IF ( cond ) stat_ (18)  
       stat : IF ( cond ) stat_ELSE stat  
  
       ELSE    shift 45  
       .       reduce 18
where the first line describes the conflict -- giving the state and the input symbol. The ordinary state description gives the grammar rules active in the state and the parser actions. Recall that the underscore marks the portion of the grammar rules that has been seen.
Thus in the example, in state 23, the parser has seen input corresponding to:

  IF ( cond ) stat
and the two grammar rules shown are active at this time. The parser can do two possible things. If the input symbol is ELSE, it is possible to shift into state 45.
State 45 will have, as part of its description, the line:

  stat : IF ( cond ) stat ELSE_stat
because the ELSE will have been shifted in this state. In state 23, the alternative action (specified by .) is to be done if the input symbol is not mentioned explicitly in the actions. In this case, if the input symbol is not ELSE, the parser reduces to:

  stat : IF '(' cond ')' stat
by grammar rule 18.
Once again, notice that the numbers following shift commands refer to other states, while the numbers following reduce commands refer to grammar rule numbers. In the y.output file, rule numbers are printed in parentheses after those rules that can be reduced. In most states, there is a reduce action possible, and reduce is the default command. If you encounter unexpected shift-reduce conflicts, look at the -v output to decide whether the default actions are appropriate.

Precedence

There is one common situation where the rules for resolving conflicts are not sufficient. This is in the parsing of arithmetic expressions. Most of the commonly used constructions for arithmetic expressions can be naturally described by the notion of precedence levels for operators, with information about left or right associativity. Ambiguous grammars with appropriate disambiguating rules can be used to create parsers that are faster and easier to write than parsers constructed from unambiguous grammars. The basic notion is to write grammar rules of the form:

  expr : expr OP expr
and:

  expr : UNARY expr
for all binary and unary operators desired. This creates a very ambiguous grammar with many parsing conflicts.
You specify as disambiguating rules the precedence or binding strength of all the operators and the associativity of the binary operators. This information is sufficient to allow yacc to resolve the parsing conflicts in accordance with these rules and construct a parser that realizes the desired precedences and associativities.
The precedences and associativities are attached to tokens in the declarations section. This is done by a series of lines beginning with the yacc keywords %left, %right, or %nonassoc, followed by a list of tokens. All of the tokens on the same line are assumed to have the same precedence level and associativity; the lines are listed in order of increasing precedence or binding strength. Thus:

  %left '+' '-'  
  %left '*' '/'
describes the precedence and associativity of the four arithmetic operators. + and - are left associative and have lower precedence than * and /, which are also left associative. The keyword %right is used to describe right associative operators. The keyword %nonassoc is used to describe operators, like the operator .LT. in FORTRAN, that may not associate with themselves. That is, because:

  A .LT.  B .LT.  C
is illegal in FORTRAN, .LT. would be described with the keyword %nonassoc in yacc.
As an example of the behavior of these declarations, the description

  %right '='  
  %left '+' '-'  
  %left '*' '/'  
  
  %%  
  
  expr    : expr '=' expr  
           | expr '+' expr  
           | expr '-' expr  
           | expr '*' expr  
           | expr '/' expr  
           | NAME  
           ;
might be used to structure the input

  a = b = c * d - e - f * g
as follows

  a = ( b = ( ((c * d) - e) - (f * g) ) )
in order to achieve the correct precedence of operators. When this mechanism is used, unary operators must, in general, be given a precedence. Sometimes a unary operator and a binary operator have the same symbolic representation but different precedences. An example is unary and binary minus.
Unary minus may be given the same strength as multiplication, or even higher, while binary minus has a lower strength than multiplication. The keyword %prec changes the precedence level associated with a particular grammar rule. %prec appears immediately after the body of the grammar rule, before the action or closing semicolon, and is followed by a token name or literal. It causes the precedence of the grammar rule to become that of the following token name or literal. For example, the rules:

  %left '+' '-'  
  %left '*' '/'  
  
  %%  
  
  expr    : expr '+' expr  
           | expr '-' expr  
           | expr '*' expr  
           | expr '/' expr  
           | '-' expr %prec '*'  
           | NAME  
           ;
might be used to give unary minus the same precedence as multiplication.
A token declared by %left, %right, and %nonassoc need not, but may, be declared by %token as well.
Precedences and associativities are used by yacc to resolve parsing conflicts. They give rise to the following disambiguating rules:
  1. Precedences and associativities are recorded for those tokens and literals that have them.

  1. A precedence and associativity is associated with each grammar rule. It is the precedence and associativity of the final token or literal in the body of the rule. If the %prec construction is used, it overrides this default value. Some grammar rules may have no precedence and associativity associated with them.

  2. When there is a reduce-reduce or shift-reduce conflict, and either the input symbol or the grammar rule has no precedence and associativity, then the two default disambiguating rules given in the preceding section are used, and the conflicts are reported.

  3. If there is a shift-reduce conflict and both the grammar rule and the input character have precedence and associativity associated with them, then the conflict is resolved in favor of the action -- shift or reduce -- associated with the higher precedence. If precedences are equal, then associativity is used. Left associative implies reduce; right associative implies shift; nonassociating implies error.

Conflicts resolved by precedence are not counted in the number of shift-reduce and reduce-reduce conflicts reported by yacc. This means that mistakes in the specification of precedences may disguise errors in the input grammar. It is a good idea to be sparing with precedences and use them strictly until some experience has been gained. The y.output file is useful in deciding whether the parser is actually doing what was intended.
To illustrate further how you might use the precedence keywords to resolve a shift-reduce conflict, look at an example similar to the one described in the previous section. Consider the following C statement:

  if (flag) if (anotherflag) x = 1;  
       else x = 2;
The problem for the parser is whether the else goes with the first or the second if. C programmers will recognize that the else goes with the second if, contrary to to what the misleading indentation suggests. The following yacc grammar for an if-then-else construct abstracts the problem. That is, the input iises will model these C statements.

  %{  
  #include <stdio.h>  
  %}  
  %token SIMPLE IF ELSE  
  %%  
  S       ;   stmnt  
           ;  
  stmnt       : SIMPLE  
           | if_stmnt  
           ;  
  if_stmnt        : IF stmnt  
               { printf("simple if\n");}  
           | IF stmnt ELSE stmnt  
               { printf("if_then_else\n");}  
           ;  
  %%  
  int  
  yylex() {  
       int c;  
       c=getchar();  
       if (c= =EOF) return 0;  
       else switch(c) {  
           case 'i': return IF;  
           case 's': return SIMPLE;  
           case 'e': return ELSE;  
           default: return c;  
       }  
  }
When the specification is passed to yacc, however, we get the following message:

  conflicts: 1 shift/reduce
The problem is that when yacc has read iis in trying to match iises, it has two choices: recognize is as a statement (reduce) or read some more input (shift) and eventually recognize ises as a statement.
One way to resolve the problem is to invent a new token REDUCE, whose sole purpose is to give the correct precedence for the rules:

  %{  
  #include <stdio.h>  
  %}  
  %token SIMPLE IF  
  %nonassoc REDUCE  
  %nonassoc ELSE  
  %%  
  S       : stmnt '\n'  
           ;  
  stmnt       : SIMPLE  
           | if_stmnt  
           ;  
  if_stmnt    : IF stmnt %prec REDUCE  
           { printf("simple if");}  
       | IF stmnt ELSE stmnt  
           { printf("if_then_else");}  
       ;  
  %%
Since the precedence associated with the second form of if_stmnt is higher now, yacc will try to match that rule first, and no conflict will be reported.
Actually, in this simple case, the new token is not needed:

  %nonassoc IF  
  %nonassoc ELSE
would also work. Moreover, it is not really necessary to resolve the conflict in this way, because, as we have seen, yacc will shift by default in a shift-reduce conflict. Resolving conflicts is a good idea, though, in the sense that you should not see diagnostic messages for correct specifications.

Error Handling

Error handling contains many semantic problems. When an error is found, for example, it may be necessary to reclaim parse tree storage, delete or alter symbol table entries, and, typically, set switches to avoid generating any further output.
It is seldom acceptable to stop all processing when an error is found. It is more useful to continue scanning the input to find further syntax errors. This leads to the problem of getting the parser restarted after an error. A general class of algorithms to do this involves discarding a number of tokens from the input string and attempting to adjust the parser so that input can continue.
To allow the user some control over this process, yacc provides the token name error. This name can be used in grammar rules. In effect, it suggests where errors are expected and recovery might take place.
The parser pops its stack until it enters a state where the token error is legal. It then behaves as if the token error were the current lookahead token and performs the action encountered. The lookahead token is then reset to the token that caused the error. If no special error rules have been specified, the processing halts when an error is detected.
To prevent a cascade of error messages, the parser, after detecting an error, remains in error state until three tokens have been successfully read and shifted. If an error is detected when the parser is already in error state, no message is given, and the input token is deleted.
As an example, a rule of the form:

  stat : error
means that on a syntax error the parser attempts to skip over the statement in which the error is seen. More precisely, the parser scans ahead, looking for three tokens that might legally follow a statement, and starts processing at the first of these. If the beginnings of statements are not sufficiently distinctive, it may make a false start in the middle of a statement and end up reporting a second error where there is in fact no error.
Actions may be used with these special error rules. These actions might attempt to reinitialize tables, reclaim symbol table space, and so forth.
Error rules such as the above are very general but difficult to control.
Rules such as the following are somewhat easier:

  stat : error ';'
Here, when there is an error, the parser attempts to skip over the statement but does so by skipping to the next semicolon. All tokens after the error and before the next semicolon cannot be shifted and are discarded. When the semicolon is seen, this rule will be reduced and any cleanup action associated with it performed.
Another form of error rule arises in interactive applications where it may be desirable to permit a line to be reentered after an error. The following example:

  input   : error '\n'  
               {  
                   (void) printf("Reenter last line: " );  
               }  
               input  
           {  
               $$ = $4;  
           }  
           ;
is one way to do this. There is one potential difficulty with this approach. The parser must correctly process three input tokens before it admits that it has correctly resynchronized after the error. If the reentered line contains an error in the first two tokens, the parser deletes the offending tokens and gives no message. This is unacceptable.
For this reason, there is a mechanism that can force the parser to believe that error recovery has been accomplished. The statement:

  yyerrok ;
in an action resets the parser to its normal mode. The last example can be rewritten as

  input   : error '\n'  
               {  
                   yyerrok;  
                   (void) printf("Reenter last line: " );  
               }  
               input  
           {  
               $$ = $4;  
           }  
           ;
As previously mentioned, the token seen immediately after the error symbol is the input token at which the error was discovered. Sometimes this is inappropriate; for example, an error recovery action might take upon itself the job of finding the correct place to resume input. In this case, the previous lookahead token must be cleared. The statement:

  yyclearin ;
in an action will have this effect. For example, suppose the action after error were to call some sophisticated resynchronization routine (supplied by the user) that attempted to advance the input to the beginning of the next valid statement. After this routine is called, the next token returned by yylex() is presumably the first token in a legal statement. The old illegal token must be discarded and the error state reset. A rule similar to:

  stat    : error  
           {  
               resynch();  
               yyerrok ;  
               yyclearin;  
           }  
           ;
could perform this.
These mechanisms are admittedly crude but they do allow a simple, fairly effective recovery of the parser from many errors. Moreover, the user can get control to deal with the error actions required by other portions of the program.

The yacc Environment

You create a yacc parser with the command:

  $ yacc grammar.y
where grammar.y is the file containing your yacc specification. (The .y suffix is a convention recognized by other operating system commands. It is not strictly necessary.) The output is a file of C-language subroutines called y.tab.c. The function produced by yacc is called yyparse(), and is integer-valued.
When it is called, it repeatedly calls yylex(), the lexical analyzer supplied by the user (see Lexical Analysis ), to obtain input tokens. Eventually, an error is detected, yyparse() returns the value 1, and no error recovery is possible, or the lexical analyzer returns the end-marker token and the parser accepts. In this case, yyparse() returns the value 0.
You must provide a certain amount of environment for this parser in order to obtain a working program. For example, as with every C-language program, a routine called main() must be defined that eventually calls yyparse(). In addition, a routine called yyerror() is needed to print a message when a syntax error is detected.
These two routines must be supplied in one form or another by the user. To ease the initial effort of using yacc, a library has been provided with default versions of main() and yyerror(). The library is accessed by a -ly argument to the cc command. The source codes:

  main()  
  {  
       return (yyparse());  
  }
and:

  # include <stdio.h>  
  
  yyerror(s)  
  char *s;  
  {  
       (void) fprintf(stderr, "%s\n", s);  
  }
show the triviality of these default programs. The argument to yyerror() is a string containing an error message, usually the string syntax error. The average application wants to do better than this. Ordinarily, the program should keep track of the input line number and print it along with the message when a syntax error is detected. The external integer variable yychar contains the lookahead token number at the time the error was detected (a feature which gives better diagnostic)s. Since the main() routine is probably supplied by the user (to read arguments, for instance), the yacc library is useful only in small projects or in the earliest stages of larger ones.
The external integer variable yydebug is normally set to 0. If it is set to a nonzero value, the parser will output a verbose description of its actions, including a discussion of the input symbols read and what the parser actions are.

Hints for Preparing Specifications

This part contains miscellaneous hints on preparing efficient, easy to change, and clear specifications.

Input Style

It is difficult to provide rules with substantial actions and still have a readable specification file. The following are a few style hints.
Example 1 below is written following this style, as are the examples in this section (where space permits). The central problem is to make the rules visible through the maze of action code.

Left Recursion

The algorithm used by the yacc parser encourages so-called left recursive grammar rules. Rules of the following form match this algorithm:

  name : name rest_of_rule ;
Rules such as:

  list    : item  
           | list ',' item  
           ;
and:

  seq     : item  
           | seq item  
           ;
frequently arise when writing specifications of sequences and lists. In each of these cases, the first rule will be reduced for the first item only; and the second rule will be reduced for the second and all succeeding items.
With right-recursive rules, such as:

  seq     : item  
           | item seq  
           ;
the parser is a bit bigger; the items are seen and reduced from right to left. More seriously, an internal stack in the parser is in danger of overflowing if an extremely long sequence is read (although yacc can now process very large stacks). Thus, you should use left recursion wherever reasonable.
It is worth considering if a sequence with zero elements has any meaning, and if so, consider writing the sequence specification as:

  seq     : /* empty */  
           | seq item  
           ;
using an empty rule. Once again, the first rule would always be reduced exactly once before the first item was read, and then the second rule would be reduced once for each item read. Permitting empty sequences often leads to increased generality. However, conflicts might arise if yacc is asked to decide which empty sequence it has seen when it hasn't seen enough to know.

C++ Mangled Symbols

The material for this section is an exact duplication of material found in the C++ Mangled Symbols section of Chapter 2, lex-Lexical Analysis. Please substitute yacc when they refer to lex.

Lexical Tie-Ins

Some lexical decisions depend on context. For example, the lexical analyzer might delete blanks normally, but not within quoted strings, or names might be entered into a symbol table in declarations but not in expressions. One way of handling these situations is to create a global flag that is examined by the lexical analyzer and set by actions. For example,

  %{  
       int dflag;  
  %}  
  ...  other declarations ...  
  %%  
  prog    : decls stats  
           ;  
  decls   :       /* empty */  
           {  
               dflag = 1;  
           }  
           | decls declaration  
           ;  
  
  stats   :       /* empty */  
           {  
               dflag = 0;  
           }  
           | stats statement  
           ;  
  
  .  .  .  other rules .  .  .
specifies a program consisting of zero or more declarations followed by zero or more statements. The flag dflag is now 0 when reading statements and 1 when reading declarations, except for the first token in the first statement.
This token must be seen by the parser before it can tell that the declaration section has ended and the statements have begun. In many cases, this single token exception does not affect the lexical scan. This approach represents a way of doing some things that are difficult, if not impossible, to do otherwise.

Reserved Words

Some programming languages permit you to use words like if, which are normally reserved as label or variable names, provided that such use does not conflict with the legal use of these names in the programming language. This is extremely hard to do in the framework of yacc.
It is difficult to pass information to the lexical analyzer telling it this instance of if is a keyword and that instance is a variable. Using the information found in the previous section, Lexical Tie-Ins might prove useful here.

Advanced Topics

This part discusses a number of advanced features of yacc.

Simulating error and accept in Actions

The parsing actions of error and accept can be simulated in an action by use of macros YYACCEPT and YYERROR. The YYACCEPT macro causes yyparse() to return the value 0; YYERROR causes the parser to behave as if the current input symbol had been a syntax error; yyerror() is called, and error recovery takes place.
These mechanisms can be used to simulate parsers with multiple end-markers or context-sensitive syntax checking.

Accessing Values in Enclosing Rules

An action may refer to values returned by actions to the left of the current rule. The mechanism is the same as ordinary actions, $ followed by a digit.

  sent    : adj noun verb adj noun  
           {  
               look at the sentence ...  
           }  
           ;  
  adj     : THE  
           {  
               $$ = THE;  
           }  
           | YOUNG  
           {  
               $$ = YOUNG;  
           }  
           ...  
           ;  
  noun    : DOG  
           {  
               $$ = DOG;  
           }  
           | CRONE  
           {  
               if ( $0 = = YOUNG )  
               {  
                   (void) printf( "what?\n" );  
               }  
               $$ = CRONE;  
           }  
           ;  
  ...
In this case, the digit may be 0 or negative. In the action following the word CRONE, a check is made that the preceding token shifted was not YOUNG. Note, however that this is only possible when a great deal is known about what might precede the symbol noun in the input. Nevertheless, at times this mechanism prevents a great deal of trouble, especially when a few combinations are to be excluded from an otherwise regular structure.

Support for Arbitrary Value Types

By default, the values returned by actions and the lexical analyzer are integers. yacc can also support values of other types including structures. In addition, yacc keeps track of the types and inserts appropriate union member names so that the resulting parser is strictly type checked. The yacc value stack is declared to be a union of the various types of values desired. You declare the union and associate union member names with each token and nonterminal symbol having a value. When the value is referenced through a $$ or $n construction, yacc will automatically insert the appropriate union name so that no unwanted conversions take place.
Three mechanisms provide for this typing. First, there is a way of defining the union. This must be done by the user since other subroutines, notably the lexical analyzer, must know about the union member names. Second, there is a way of associating a union member name with tokens and nonterminals. Finally, there is a mechanism for describing the type of those few values where yacc cannot easily determine the type.
To declare the union, you include:

  %union  
  {  
       body of union  
  }
in the declaration section. This declares the yacc value stack and the external variables yylval and yyval to have type equal to this union. If yacc was invoked with the -d option, the union declaration is copied into the y.tab.h file as YYSTYPE.
Once YYSTYPE is defined, the union member names must be associated with the various terminal and nonterminal names. The construction:

  <name>
is used to indicate a union member name. If this follows one of the keywords %token, %left, %right, and %nonassoc, the union member name is associated with the tokens listed.
Thus, saying

  %left <optype> '+' '-'
causes any reference to values returned by these two tokens to be tagged with the union member name optype. Another keyword, %type, is used to associate union member names with nonterminals. You could use the rule

  %type <nodetype> expr stat
to associate the union member nodetype with the nonterminal symbols expr and stat.
There remain a couple of cases where these mechanisms are insufficient. If there is an action within a rule, the value returned by this action has no a priori type. Similarly, reference to left context values (such as $0) leaves yacc with no easy way of knowing the type. In this case, a type can be imposed on the reference by inserting a union member name between < and > immediately after the first $. The example below:

  rule    : aaa  
                   {  
                       $<intval>$ = 3;  
                   }  
                   bbb  
           {  
               fun( $<intval>2, $<other>0 );  
           }  
           ;
shows this usage. This syntax has little to recommend it, but the situation arises rarely.
A sample specification is given in the An Advanced Example section. The facilities in this subsection are not triggered until they are used. In particular, the use of %type will turn on these mechanisms. When they are used, there is a fairly strict level of checking.
For example, use of $n or $$ to refer to something with no defined type is diagnosed. If these facilities are not triggered, the yacc value stack is used to hold ints.

yacc Input Syntax

This section has a description of the yacc input syntax as a yacc specification. Context dependencies and so forth are not considered. Although yacc accepts an LALR(1) grammar, the yacc input specification language is specified as an LR(2) grammar; thedifficulty arises when an identifier is seen in a rule immediately following an action.
If this identifier is followed by a colon, it is the start of the next rule; otherwise, it is a continuation of the current rule, which just happens to have an action embedded in it. As implemented, the lexical analyzer looks ahead after seeing an identifier and figures out whether the next token (skipping blanks, newlines, comments, and so on) is a colon. If so, it returns the token
C_IDENTIFIER.
Otherwise, it returns IDENTIFIER. Literals (quoted strings) are also returned as IDENTIFIERs but never as part of C_IDENTIFIERs.
Figure 3-1 The yacc Input Syntax

                           /* grammar for the input to yacc */  
                           /* basic entries */  
  %token IDENTIFIER       /* includes identifiers and literals */  
  %token C_IDENTIFIER /* identifier (but not literal) */  
                           /* followed by a : */  
  %token NUMBER           /* [0-9]+ */  
                   /* reserved words: %type=>TYPE %left=>LEFT,etc.  */  
  %token LEFT RIGHT NONASSOC TOKEN PREC TYPE START UNION  
  %token MARK             /* the %% mark */
Figure 3-1 The yacc Input Syntax

  %token LCURL            /* the %{ mark */  
  %token RCURL            /* the %} mark */  
                   /* ASCII character literals stand for themselves */  
  
  %token spec t  
  
  %%  
  
  spec    : defs MARK rules tail  
           ;  
  tail    : MARK  
           {  
               In this action,read in the rest of the file  
           }  
           |           /* empty: the second MARK is optional */  
           ;  
  defs    :           /* empty */  
           | defs def  
           ;  
  def     : START IDENTIFIER  
           | UNION  
           {  
               Copy union definition to output  
           }  
           | LCURL
Figure 3-1 The yacc Input Syntax

           {  
               Copy C code to output file  
           }  
               RCURL  
           | rword tag nlist  
           ;  
  rword   : TOKEN  
           | LEFT  
           | RIGHT  
           | NONASSOC  
           | TYPE  
           ;  
  tag     :                       /* empty: union tag is optional */  
           | '<' IDENTIFIER '>'  
           ;  
  
  nlist   : nmno  
           | nlist nmno  
           | nlist ',' nmno  
           ;  
  nmno    : IDENTIFIER        /* Note: literal illegal with % type */  
           | IDENTIFIER NUMBER         /* Note: illegal with % type */  
           ;  
  
                       /* rule section */
Figure 3-1 The yacc Input Syntax

  rules   : C_IDENTIFIER rbody prec  
           | rules rule  
           ;  
  rule    : C_IDENTIFIER rbody prec  
           | '|' rbody prec  
           ;  
  
  rbody   : /* empty */  
           | rbody IDENTIFIER  
           | rbody act  
           ;  
  
  act     : '{'  
           {  
               Copy action translate $$ etc.  
           }  
           '}'  
           ;  
  
  prec    : /* empty */  
           | PREC IDENTIFIER  
           | PREC IDENTIFIER act  
           | prec ';'  
           ;

Examples

A Simple Example

Figure 3-2 shows the complete yacc applications for a small desk calculator. The calculator has 26 registers labeled a through z and accepts arithmetic expressions made up of the operators +, -, *, /, %, &, |, and the assignment operators.
If an expression at the top level is an assignment, only the assignment is made; otherwise, the expression is printed. As in the C language, an integer that begins with 0 is assumed to be octal; otherwise, it is assumed to be decimal.
As an example of a yacc specification, the desk calculator shows how precedence and ambiguities are used and demonstrates simple recovery. The major oversimplifications are that the lexical analyzer is much simpler than for most applications, and the output is produced immediately, line by line.
Note the way that decimal and octal integers are read by grammar rules. This jobcan also be performed by the lexical analyzer.
Figure 3-2 A yacc Application for a Desk Calculator

  %{  
  # include <stdio.h>  
  # include <ctype.h>  
  
  int regs[26];  
  int base;  
  
  %}  
  
  %start list  
  %token DIGIT LETTER
Figure 3-2 A yacc Application for a Desk Calculator

  %left '|'  
  %left '&'  
  %left '+' '-'  
  %left '*' '/' '%'  
  %left UMINUS            /* supplies precedence for unary minus */  
  
  %%                      /* beginning of rules section */  
  
  list    :               /* empty */  
           | list stat '\n'  
           | list error '\n'  
           {  
               yyerrok;  
           }  
           ;  
  
  stat    : expr  
           {  
               (void) printf( "%d\n", $1 );  
           }  
           | LETTER '=' expr  
           {  
               regs[$1] = $3;  
           }  
           ;
Figure 3-2 A yacc Application for a Desk Calculator

  expr    : '(' expr ')'  
           {  
               $$ = $2;  
           }  
           | expr '+' expr  
           {  
               $$ = $1 + $3;  
           }  
           | expr '-' expr  
           {  
               $$ = $1 - $3;  
           {  
           | expr '*' expr  
           {  
               $$ = $1 * $3;  
           }  
           | expr '/' expr  
           {  
               $$ = $1 / $3;  
           }  
           | exp '%' expr  
           {  
               $$ = $1 % $3;  
           }
Figure 3-2 A yacc Application for a Desk Calculator

           | expr '&' expr  
           {  
               $$ = $1 & $3;  
           }  
           | expr '|' expr  
           {  
               $$ = $1 | $3;  
           }  
           | '-' expr %prec UMINUS  
           {  
               $$ = -$2;  
           }  
           | LETTER  
           {  
               $$ = reg[$1];  
           }  
           | number  
           ;  
  
  number : DIGIT  
           {  
               $$ = $1; base = ($1= =0) ? 8 ; 10;  
           }  
           | number DIGIT  
           {
Figure 3-2 A yacc Application for a Desk Calculator

               $$ = base * $1 + $2;  
           }  
           ;  
  
  %%                      /* beginning of subroutines section */  
  
  int yylex( )                /* lexical analysis routine */  
  {                           /* return LETTER for lowercase letter, */  
                               /* yylval = 0 through 25 returns DIGIT */  
                               /* for digit, yylval = 0 through 9 */  
                   /* all other characters are returned immediately */  
       int c;  
                       /*skip blanks*/  
       while ((c = getchar()) = = ' ')  
           ;  
  
                       /* c is now nonblank */  
  
       if (islower(c)) {  
           yylval = c - 'a';  
           return (LETTER);  
       }  
       if (isdigit(c)) {  
           yylval = c - '0';  
           return (DIGIT);
Figure 3-2 A yacc Application for a Desk Calculator

       }  
       return (c);  
  }

An Advanced Example

This section gives an example of a grammar using some of the advanced features. The desk calculator in Example 1 is modified to provide a desk calculator that does floating point interval arithmetic. The calculator understands floating point constants, and the arithmetic operations +, -, *, /, and unary -. It uses the registers a through z. Moreover, it understands intervals written

  (X,Y)
where X is less than or equal to Y. There are 26 interval valued variables A through Z that may also be used. The usage is similar to that in Example 1; assignments return no value and print nothing while expressions print the (floating or interval) value.
This example explores a number of features of yacc and C. Intervals are represented by a structure consisting of the left and right endpoint values stored as doubles. This structure is given a type name, INTERVAL, by means of typedef.
The yacc value stack can also contain floating point scalars and integers (used to index into the arrays holding the variable values). Notice that the entire strategy depends strongly on being able to assign structures and unions in C language. In fact, many of the actions call functions that return structures as well.
Note the use of YYERROR to handle error conditions -- division by an interval containing 0 and an interval presented in the wrong order. The error recovery mechanism of yacc is used to throw away the rest of the offending line.
In addition to the mixing of types on the value stack, this grammar also demonstrates a syntax to keep track of the type (for example, scalar or interval) of intermediate expressions. Note that scalar-value can be automatically promoted to an interval if the context demands an interval value. This causes
a large number of conflicts when the grammar is run through yacc: 18 shift-reduce and 26 reduce-reduce. The problem can be seen by looking at the two input lines:

  2.5 + (3.5 - 4.)
and:

  2.5 + (3.5, 4)
Notice that the 2.5 is to be used in an interval value expression in the second example, but this fact is not known until the comma is read. By this time, 2.5 is finished, and the parser cannot go back and do something else. More generally, it might be necessary to look ahead an arbitrary number of tokens to decide whether to convert a scalar to an interval.
This problem is evaded by having two rules for each binary interval valued operator -- one when the left operand is a scalar and one when the left operand is an interval. In the second case, the right operand must be an interval, so the conversion will be applied automatically.
Despite this evasion, there are still many cases where the conversion may be applied or not, leading to the above conflicts. They are resolved by listing the rules that yield scalars first in the specification file; in this way, the conflict will be resolved in the direction of keeping scalar-valued expressions scalar valued until they are forced to become intervals.
This way of handling multiple types is instructive. If there were many kinds of expression types instead of just two, the number of rules needed would increase dramatically and the conflicts even more dramatically.
Thus, while this example is instructive, it is better practice in a more normal programming language environment to keep the type information as part of the value and not as part of the grammar.
Finally, a word about the lexical analysis. The only unusual feature is the treatment of floating-point constants. The C-language library routine atof() is used to do the actual conversion from a character string to a double-precision value. If the lexical analyzer detects an error, it responds by returning a token that is illegal in the grammar, provoking a syntax error in the parser and thence error recovery. The following, Figure 3-3, is a yacc Specification.
Figure 3-3 Advanced Example of a yacc Specification

  %{  
  #include <stdio.h>  
  #include <ctype.h>  
  typedef struct interval {  
       double lo, hi;  
  } INTERVAL;  
  INTERVAL vmul(), vdiv();  
  double atof();  
  double dreg[26];  
  INTERVAL vreg[26];  
  %}  
  %start lines  
  %union {  
       int ival;  
       double dval;  
       INTERVAL vval;
Figure 3-3 Advanced Example of a yacc Specification

  }  
  
  %token <ival> DREG VREG         /* indices into dreg, vreg arrays */  
  %token <dval> CONST             /* floating point constant */  
  %type <dval> dexp               /* expression */  
  %type <vval> vexp               /* interval expression */  
  
                       /* precedence information about the operators */  
  
  %left '+' '/-'  
  %left '*' '/'  
  
  %%               /* beginning of rules section */  
  
  lines   : /* empty */  
           | lines line  
           ;  
  line    : dexp '\n'  
           {  
               (void)printf("%15.8f\n", $1);  
           }  
           | vexp '\n'  
           {  
               (void)printf("(%15.8f, %15.8f)\n", $1.lo, $1.hi);  
           }
Figure 3-3 Advanced Example of a yacc Specification

           | DREG '=' dexp '\n'  
           {  
               dreg[$1] = $3;  
           }  
           | VREG '=' vexp '\n'  
           {  
               vreg[$1] = $3;  
           }  
           | error '\n'  
           {  
               yyerrok;  
           }  
           ;  
  dexp    : CONST  
           | DREG  
           {  
               $$ = dreg[$1];  
           }  
           | dexp '+' dexp  
           {  
               $$ = $1 + $3;  
           }  
           | dexp '-' dexp  
           {  
               $$ = $1 - $3;
Figure 3-3 Advanced Example of a yacc Specification

           }  
           | dexp '*' dexp  
           {  
               $$ = $1 * $3;  
           }  
           | dexp '/' dexp  
           {  
               $$ = $1 / $3;  
           }  
           | '-' dexp  
           {  
               $$ = -$2;  
           }  
           | '(' dexp ')'  
           {  
               $$ = $2;  
           }  
           ;  
  vexp    : dexp  
           {  
               $$.hi = $$.lo = $1;  
           }  
           | '(' dexp ',' dexp ')'  
           {  
               $$.lo = $2;
Figure 3-3 Advanced Example of a yacc Specification

               $$.hi = $4;  
               if($$.lo > $$.hi) {  
                   (void) printf("interval out of order\n");  
                   YYERROR;  
               }  
           }  
           | VREG  
           {  
               $$ = vreg[$1];  
           }  
           | vexp '+' vexp  
           {  
               $$.hi = $1.hi + $3.hi;  
               $$.lo = $1.lo + $3.lo;  
           }  
           | dexp '+' vexp  
           {  
               $$.hi = $1 + $3.hi;  
               $$.lo = $1 + $3.lo;  
           }  
           | vexp '-' vexp  
           {  
               $$.hi = $1.hi - $3.lo;  
               $$.lo = $1.lo - $3.hi;  
           }
Figure 3-3 Advanced Example of a yacc Specification

           | dexp '-' vexp  
           {  
               $$.hi = $1 - $3.lo;  
               $$.lo = $1 - $3.hi;  
           }  
           | vexp '*' vexp  
           {  
               $$ = vmul($1.lo, $1.hi, $3);  
           }  
           | dexp '*' vexp  
           {  
               $$ = vmul($1, $1, $3);  
           }  
           | vexp '/' vexp  
           {  
               if (dcheck($3)) YYERROR;  
               $$ = vdiv($1.lo, $1.hi, $3);  
           }  
           | dexp '/' vexp  
           {  
               if (dcheck($3)) YYERROR;  
               $$ = vdiv($1, $1, $3);  
           }  
           | '-' vexp  
           {
Figure 3-3 Advanced Example of a yacc Specification

               $$.hi = -$2.lo; $$.lo = -$2.hi;  
           }  
           | '(' vexp ')'  
           {  
               $$ = $2;  
           }  
           ;  
  
  %%                      /* beginning of subroutines section */  
  
  # define BSZ 50         /* buffer size for floating point number */  
  
  /* lexical analysis */  
  
  int yylex()  
  {  
       register int c;  
                       /* skip over blanks */  
       while ((c=getchar()) = = ' ')  
               ;  
       if (isupper(c)) {  
           yylval.ival = c - 'A';  
           return(VREG);  
       }  
       if (islower(c)) {
Figure 3-3 Advanced Example of a yacc Specification

           yylval.ival = c - 'a';  
           return(DREG);  
       }  
                   /* digits, points, exponents */  
       if (isdigit(c) || c = = '.') {  
           char buf[BSZ + 1], *cp = buf;  
           int dot = 0, exp = 0;  
           for (;(cp - buf) < BSZ; ++cp, c = getchar()) {  
               *cp = c;  
               if (isdigit(c))  
                   continue;  
               if (c = = '.') {  
                   if (dot++ || exp)  
                       return('.');        /* will cause syntax error */  
  
                   continue;  
               }  
               if (c = = 'e') {  
  
                   if (exp++)  
                       return('e');        /* will cause syntax error */  
  
                   continue;  
               }  
                           /* end of number */
Figure 3-3 Advanced Example of a yacc Specification

               break;  
           }  
  
           *cp = '\0';  
           if (cp - buf >= BSZ)  
               (void)printf("constant too long -- truncated\n");  
           else  
               ungetc(c, stdin);       /* push back last char read */  
           yylval.dval = atof(buf);  
           return(CONST);  
       }  
       return(c);  
  }  
  
  INTERVAL  
  hilo(a, b, c, d)  
  double a, b, c, d;  
  {  
           /* returns the smallest interval containing a, b, c, and d  
           used by vmul, vdiv routines */  
  
       INTERVAL v;  
  
       if (a > b){
Figure 3-3 Advanced Example of a yacc Specification

           v.hi = a;  
           v.lo = b;  
       }  
       else{  
  
           v.hi = b;  
           v.lo = a;  
       }  
       if (c > d) {  
  
           if (c > v.hi)  
               v.hi = c;  
           if (d < v.lo)  
               v.lo = d;  
       }  
       else {  
  
           if (d > v.hi)  
               v.hi = d;  
           if (c < v.lo)  
               v.lo = c;  
       }  
       return(v);  
  }
Figure 3-3 Advanced Example of a yacc Specification

  INTERVAL  
  vmul(a, b, v)  
  double a, b;  
  INTERVAL v;  
  {  
       return(hilo(a * v.hi, a * v.lo, b * v.hi, b * v.lo));  
  }  
  
  dcheck(v)  
  INTERVAL v;  
  {  
       if (v.hi >= 0.  && v.lo <= 0.) {  
  
           (void) printf("divisor interval contains 0.\n");  
           return(1);  
       }  
  return(0);  
  }  
  
  INTERVAL  
  vdiv(a, b, v)  
  double a, b;  
  INTERVAL v;  
  {  
       return(hilo(a / v.hi, a / v.lo, b / v.hi, b / v.lo));

Next Topic

Book Table of ContentsComplete Table of Contents for book

Copyright 1997 Sun Microsystems, Inc. All rights reserved.
Comments on: Programming Utilities Guide