UNIT-II SYNTAX ANALYSIS AND RUNTIME ENVIRONMENT SYNTAX ANALYSIS Syntax analysis is the second phase of the compiler.

It gets the input from the tokens and generates a syntax tree or parse tree. Advantages of grammar for syntactic specification : 1. A grammar gives a precise and easy-to-understand syntactic specification of a programming language. 2. An efficient parser can be constructed automatically from a properly designed grammar. 3. A grammar imparts a structure to a source program that is useful for its translation into object code and for the detection of errors. 4. New constructs can be added to a language more easily when there is a grammatical description of the language. THE ROLE OF PARSER The parser or syntactic analyzer obtains a string of tokens from the lexical analyzer and verifies that the string can be generated by the grammar for the source language. It reports any syntax errors in the program. It also recovers from commonly occurring errors so that it can continue processing its input. Position of parser in compiler model source program lexical analyzer token get next token parser parse tree rest of front end intermediate representation

symbol table Functions of the parser : 1. It verifies the structure generated by the tokens based on the grammar. 2. It constructs the parse tree. 3. It reports the errors. 4. It performs error recovery. Issues : Parser cannot detect errors such as:

NOTES.PMR-INSIGNIA.ORG

1. Variable re-declaration 2. Variable initialization before use. 3. Data type mismatch for an operation. The above issues are handled by Semantic Analysis phase. Syntax error handling : Programs can contain errors at many different levels. For example : 1. Lexical, such as misspelling a keyword. 2. Syntactic, such as an arithmetic expression with unbalanced parentheses. 3. Semantic, such as an operator applied to an incompatible operand. 4. Logical, such as an infinitely recursive call. Functions of error handler : 1. It should report the presence of errors clearly and accurately. 2. It should recover from each error quickly enough to be able to detect subsequent errors. 3. It should not significantly slow down the processing of correct programs. Error recovery strategies : The different strategies that a parse uses to recover from a syntactic error are: 1. Panic mode 2. Phrase level 3. Error productions 4. Global correction Panic mode recovery: On discovering an error, the parser discards input symbols one at a time until a synchronizing token is found. The synchronizing tokens are usually delimiters, such as semicolon or end. It has the advantage of simplicity and does not go into an infinite loop. When multiple errors in the same statement are rare, this method is quite useful. Phrase level recovery: On discovering an error, the parser performs local correction on the remaining input that allows it to continue. Example: Insert a missing semicolon or delete an extraneous semicolon etc. Error productions: The parser is constructed using augmented grammar with error productions. If an error production is used by the parser, appropriate error diagnostics can be generated to indicate the erroneous constructs recognized by the input. Global correction: Given an incorrect input string x and grammar G, certain algorithms can be used to find a parse tree for a string y, such that the number of insertions, deletions and changes of tokens is as small as possible. However, these methods are in general too costly in terms of time and space.

NOTES.PMR-INSIGNIA.ORG

CONTEXT-FREE GRAMMARS A Context-Free Grammar is a quadruple that consists of terminals, non-terminals, start symbol and productions. Terminals : These are the basic symbols from which strings are formed. Non-Terminals : These are the syntactic variables that denote a set of strings. These help to define the language generated by the grammar. Start Symbol : One non-terminal in the grammar is denoted as the Start-symbol and the set of strings it denotes is the language defined by the grammar. Productions : It specifies the manner in which terminals and non-terminals can be combined to form strings. Each production consists of a non-terminal, followed by an arrow, followed by a string of non-terminals and terminals. Example of context-free grammar: The following grammar defines simple arithmetic expressions: expr expr expr expr op op op op op expr op expr (expr) - expr id + * /

In this grammar, id + - * / ( ) are terminals. expr , op are non-terminals. expr is the start symbol. Each line is a production.

Derivations: Two basic requirements for a grammar are : 1. To generate a valid string. 2. To recognize a valid string. Derivation is a process that generates a valid string with the help of grammar by replacing the non-terminals on the left with the string on the right side of the production. Example : Consider the following grammar for arithmetic expressions : E E+E | E*E | ( E ) | - E | id

NOTES.PMR-INSIGNIA.ORG

To generate a valid string - ( id+id ) from the grammar the steps are 1. E - E 2. E - ( E ) 3. E - ( E+E ) 4. E - ( id+E ) 5. E - ( id+id ) In the above derivation, E is the start symbol. - (id+id) is the required sentence (only terminals). Strings such as E, -E, -(E), . . . are called sentinel forms. Types of derivations: The two types of derivation are: 1. Left most derivation 2. Right most derivation. In leftmost derivations, the leftmost non-terminal in each sentinel is always chosen first for replacement. In rightmost derivations, the rightmost non-terminal in each sentinel is always chosen first for replacement. Example: Given grammar G : E E+E | E*E | ( E ) | - E | id Sentence to be derived : (id+id) LEFTMOST DERIVATION E-E E-(E) E - ( E+E ) E - ( id+E ) E - ( id+id ) RIGHTMOST DERIVATION E-E E-(E) E - (E+E ) E - ( E+id ) E - ( id+id )

String that appear in leftmost derivation are called left sentinel forms. String that appear in rightmost derivation are called right sentinel forms. Sentinels: Given a grammar G with start symbol S, if S , where may contain non-terminals or terminals, then is called the sentinel form of G.

NOTES.PMR-INSIGNIA.ORG

Yield or frontier of tree: Each interior node of a parse tree is a non-terminal. The children of node can be a terminal or non-terminal of the sentinel forms that are read from left to right. The sentinel form in the parse tree is called yield or frontier of the tree. Ambiguity: A grammar that produces more than one parse for some sentence is said to be ambiguous grammar. Example : Given grammar G : E E+E | E*E | ( E ) | - E | id The sentence id+id*id has the following two distinct leftmost derivations: E E+ E E id + E E id + E * E E id + id * E E id + id * id The two corresponding parse trees are : E E id + E id E * E id E id E + E * E id E id E E* E EE+E*E E id + E * E E id + id * E E id + id * id

WRITING A GRAMMAR There are four categories in writing a grammar : 1. Regular Expression Vs Context Free Grammar 2. Eliminating ambiguous grammar. 3. Eliminating left-recursion 4. Left-factoring. Each parsing method can handle grammars only of a certain form hence, the initial grammar may have to be rewritten to make it parsable.

NOTES.PMR-INSIGNIA.ORG

Regular Expressions vs. Context-Free Grammars: REGULAR EXPRESSION CONTEXT-FREE GRAMMAR

It is used to describe the tokens of programming It consists of a quadruple where S start languages. symbol, P production, T terminal, V variable or non- terminal. It is used to check whether the given input is It is used to check whether the given input is valid or not using transition diagram. valid or not using derivation. The transition diagram has set of states and The context-free edges. productions. It has no start symbol. It has start symbol. grammar has set of

It is useful for describing the structure of lexical It is useful in describing nested structures constructs such as identifiers, constants, such as balanced parentheses, matching keywords, and so forth. begin-ends and so on. The lexical rules of a language are simple and RE is used to describe them. Regular expressions provide a more concise and easier to understand notation for tokens than grammars. Efficient lexical analyzers can be constructed automatically from RE than from grammars. Separating the syntactic structure of a language into lexical and nonlexical parts provides a convenient way of modularizing the front end into two manageable-sized components. Eliminating ambiguity: Ambiguity of the grammar that produces more than one parse tree for leftmost or rightmost derivation can be eliminated by re-writing the grammar. Consider this example, G: stmt if expr then stmt | if expr then stmt else stmt | other This grammar is ambiguous since the string if E1 then if E2 then S1 else S2 has the following two parse trees for leftmost derivation :

NOTES.PMR-INSIGNIA.ORG

1.

stmt

if

expr

then

stmt

E1 if expr then stmt else stmt

E2

S1

S2

2.

stmt

if

expr

then

stmt

else

stmt

E1 if expr then stmt

S2

E2

S1

To eliminate ambiguity, the following grammar may be used: stmt matched_stmt | unmatched_stmt matched_stmt if expr then matched_stmt else matched_stmt | other unmatched_stmt if expr then stmt | if expr then matched_stmt else unmatched_stmt

Eliminating Left Recursion: A grammar is said to be left recursive if it has a non-terminal A such that there is a derivation A=>A for some string . Top-down parsing methods cannot handle left-recursive grammars. Hence, left recursion can be eliminated as follows:

NOTES.PMR-INSIGNIA.ORG

If there is a production A A | it can be replaced with a sequence of two productions A A A A | without changing the set of strings derivable from A. Example : Consider the following grammar for arithmetic expressions: E E+T | T T T*F | F F (E) | id First eliminate the left recursion for E as E TE E +TE | Then eliminate for T as T FT T *FT | Thus the obtained grammar after eliminating left recursion is E TE E +TE | T FT T *FT | F (E) | id Algorithm to eliminate left recursion: 1. Arrange the non-terminals in some order A1, A2 . . . An. 2. for i := 1 to n do begin for j := 1 to i-1 do begin replace each production of the form A i Aj by the productions Ai 1 | 2 | . . . | k where Aj 1 | 2 | . . . | k are all the current Aj-productions; end eliminate the immediate left recursion among the Ai-productions end

NOTES.PMR-INSIGNIA.ORG

Left factoring: Left factoring is a grammar transformation that is useful for producing a grammar suitable for predictive parsing. When it is not clear which of two alternative productions to use to expand a non-terminal A, we can rewrite the A-productions to defer the decision until we have seen enough of the input to make the right choice. If there is any production A 1 | 2 , it can be rewritten as A A A 1 | 2 Consider the grammar , G : S iEtS | iEtSeS | a Eb Left factored, this grammar becomes S iEtSS | a S eS | Eb PARSING It is the process of analyzing a continuous stream of input in order to determine its grammatical structure with respect to a given formal grammar. Parse tree: Graphical representation of a derivation or deduction is called a parse tree. Each interior node of the parse tree is a non-terminal; the children of the node can be terminals or nonterminals. Types of parsing: 1. 2. Top down parsing Bottom up parsing Topdown parsing : A parser can start with the start symbol and try to transform it to the input string. Example : LL Parsers. Bottomup parsing : A parser can start with input and attempt to rewrite it into the start symbol. Example : LR Parsers.

TOP-DOWN PARSING It can be viewed as an attempt to find a left-most derivation for an input string or an attempt to construct a parse tree for the input starting from the root to the leaves.

NOTES.PMR-INSIGNIA.ORG

Types of top-down parsing : 1. Recursive descent parsing 2. Predictive parsing 1. RECURSIVE DESCENT PARSING Recursive descent parsing is one of the top-down parsing techniques that uses a set of recursive procedures to scan its input . This parsing method may involve backtracking, that is, making repeated scans of the input.

Example for backtracking : Consider the grammar G : S cAd A ab | a and the input string w=cad. The parse tree can be constructed using the following top-down approach : Step1: Initially create a tree with single node labeled S. An input pointer points to c, the first symbol of w. Expand the tree with the production of S. S c Step2: The leftmost leaf c matches the first symbol of w, so advance the input pointer to the second symbol of w a and consider the next leaf A. Expand A using the first alternative. S A d

a Step3:

The second symbol a of w also matches with second leaf of tree. So advance the input pointer to third symbol of w d. But the third leaf of tree is b which does not match with the input symbol d.

NOTES.PMR-INSIGNIA.ORG

Hence discard the chosen production and reset the pointer to second position. This is called backtracking. Step4: Now try the second alternative for A. S

a Now we can halt and announce the successful completion of parsing. Example for recursive decent parsing: A left-recursive grammar can cause a recursive-descent parser to go into an infinite loop. Hence, elimination of left-recursion must be done before parsing. Consider the grammar for arithmetic expressions E E+T | T T T*F | F F (E) | id After eliminating the left-recursion the grammar becomes, E TE E +TE | T FT T *FT | F (E) | id Now we can write the procedure for grammar as follows: Recursive procedure: Procedure E() begin T( ); EPRIME( ); end

NOTES.PMR-INSIGNIA.ORG

Procedure EPRIME( ) begin If input_symbol=+ then ADVANCE( ); T( ); EPRIME( ); end Procedure T( ) begin F( ); TPRIME( ); end Procedure TPRIME( ) begin If input_symbol=* then ADVANCE( ); F( ); TPRIME( ); end Procedure F( ) begin If input-symbol=id then ADVANCE( ); else if input-symbol=( then ADVANCE( ); E( ); else if input-symbol=) then ADVANCE( ); end else ERROR( ); Stack implementation: To recognize input id+id*id :
PROCEDURE INPUT STRING

E( ) T( ) F( ) ADVANCE( )

id+id*id id+id*id id+id*id id+id*id

NOTES.PMR-INSIGNIA.ORG

TPRIME( ) EPRIME( ) ADVANCE( ) T( ) F( ) ADVANCE( ) TPRIME( ) ADVANCE( ) F( ) ADVANCE( ) TPRIME( )

id+id*id id+id*id id+id*id id+id*id id+id*id id+id*id id+id*id id+id*id id+id*id id+id*id id+id*id

2. PREDICTIVE PARSING

Predictive parsing is a special case of recursive descent parsing where no backtracking is required. The key problem of predictive parsing is to determine the production to be applied for a non-terminal in case of alternatives.

Non-recursive predictive parser

INPUT

STACK

X Y Z $

Predictive parsing program

OUTPUT

Parsing Table M

NOTES.PMR-INSIGNIA.ORG

The table-driven predictive parser has an input buffer, stack, a parsing table and an output stream. Input buffer: It consists of strings to be parsed, followed by $ to indicate the end of the input string. Stack: It contains a sequence of grammar symbols preceded by $ to indicate the bottom of the stack. Initially, the stack contains the start symbol on top of $. Parsing table: It is a two-dimensional array M[A, a], where A is a non-terminal and a is a terminal. Predictive parsing program: The parser is controlled by a program that considers X, the symbol on top of stack, and a, the current input symbol. These two symbols determine the parser action. There are three possibilities: 1. If X = a = $, the parser halts and announces successful completion of parsing. 2. If X = a $, the parser pops X off the stack and advances the input pointer to the next input symbol. 3. If X is a non-terminal , the program consults entry M[X, a] of the parsing table M. This entry will either be an X-production of the grammar or an error entry. If M[X, a] = {X UVW},the parser replaces X on top of the stack by WVU. If M[X, a] = error, the parser calls an error recovery routine. Algorithm for nonrecursive predictive parsing: Input : A string w and a parsing table M for grammar G. Output : If w is in L(G), a leftmost derivation of w; otherwise, an error indication. Method : Initially, the parser has $S on the stack with S, the start symbol of G on top, and w$ in the input buffer. The program that utilizes the predictive parsing table M to produce a parse for the input is as follows: set ip to point to the first symbol of w$; repeat let X be the top stack symbol and a the symbol pointed to by ip; if X is a terminal or $ then if X = a then pop X from the stack and advance ip else error() else /* X is a non-terminal */ if M[X, a] = X Y1Y2 Yk then begin

NOTES.PMR-INSIGNIA.ORG

end else error() until X = $ /* stack is empty */ Predictive parsing table construction:

pop X from the stack; push Yk, Yk-1, ,Y1 onto the stack, with Y1 on top; output the production X Y1 Y2 . . . Yk

The construction of a predictive parser is aided by two functions associated with a grammar G : 1. FIRST 2. FOLLOW Rules for first( ): 1. If X is terminal, then FIRST(X) is {X}. 2. If X is a production, then add to FIRST(X). 3. If X is non-terminal and X a is a production then add a to FIRST(X). 4. If X is non-terminal and X Y1 Y2Yk is a production, then place a in FIRST(X) if for some i, a is in FIRST(Yi), and is in all of FIRST(Y1),,FIRST(Yi-1); that is, Y1,.Yi-1 => . If is in FIRST(Yj) for all j=1,2,..,k, then add to FIRST(X). Rules for follow( ): 1. If S is a start symbol, then FOLLOW(S) contains $. 2. If there is a production A B, then everything in FIRST() except is placed in follow(B). 3. If there is a production A B, or a production A B where FIRST() contains , then everything in FOLLOW(A) is in FOLLOW(B). Algorithm for construction of predictive parsing table: Input : Grammar G Output : Parsing table M Method : 1. For each production A of the grammar, do steps 2 and 3. 2. For each terminal a in FIRST(), add A to M[A, a]. 3. If is in FIRST(), add A to M[A, b] for each terminal b in FOLLOW(A). If is in FIRST() and $ is in FOLLOW(A) , add A to M[A, $]. 4. Make each undefined entry of M be error.

NOTES.PMR-INSIGNIA.ORG

Example: Consider the following grammar : E E+T | T T T*F | F F (E) | id After eliminating left-recursion the grammar is E TE E +TE | T FT T *FT | F (E) | id First( ) : FIRST(E) = { ( , id} FIRST(E) ={+ , } FIRST(T) = { ( , id} FIRST(T) = {*, } FIRST(F) = { ( , id } Follow( ): FOLLOW(E) = { $, ) } FOLLOW(E) = { $, ) } FOLLOW(T) = { +, $, ) } FOLLOW(T) = { +, $, ) } FOLLOW(F) = {+, * , $ , ) }

Predictive parsing table :

id E TE T FT F id

+ E +TE T

( E TE

) E T

$ E T

E E T T F

T *FT

T FT F (E)

NOTES.PMR-INSIGNIA.ORG

Stack implementation: stack $E $ET $ETF $ETid $ET $E $ET+ $ET $ETF $ETid $ET $ETF* $ETF $ETid $ET $E $ LL(1) grammar: The parsing table entries are single entries. So each location has not more than one entry. This type of grammar is called LL(1) grammar. Consider this following grammar: S iEtS | iEtSeS | a Eb After eliminating left factoring, we have S iEtSS | a S eS | Eb To construct a parsing table, we need FIRST() and FOLLOW() for all the non-terminals. FIRST(S) = { i, a } FIRST(S) = {e, } FIRST(E) = { b} FOLLOW(S) = { $ ,e } Input id+id*id $ id+id*id $ id+id*id $ id+id*id $ +id*id $ +id*id $ +id*id $ id*id $ id*id $ id*id $ *id $ *id $ id $ id $ $ $ $ Output E TE T FT F id T E +TE T FT F id T *FT F id T E

NOTES.PMR-INSIGNIA.ORG

FOLLOW(S) = { $ ,e } FOLLOW(E) = {t} Parsing table:

a Sa

e S eS S

i S iEtSS

$ S

S E Eb

Since there are more than one production, the grammar is not LL(1) grammar. Actions performed in predictive parsing: 1. 2. 3. 4. Shift Reduce Accept Error

Implementation of predictive parser: 1. 2. 3. 4. Elimination of left recursion, left factoring and ambiguous grammar. Construct FIRST() and FOLLOW() for all non-terminals. Construct predictive parsing table. Parse the given input string using stack and parsing table.

BOTTOM-UP PARSING Constructing a parse tree for an input string beginning at the leaves and going towards the root is called bottom-up parsing. A general type of bottom-up parser is a shift-reduce parser.
SHIFT-REDUCE PARSING

Shift-reduce parsing is a type of bottom-up parsing that attempts to construct a parse tree for an input string beginning at the leaves (the bottom) and working up towards the root (the top). Example: Consider the grammar: S aABe A Abc | b Bd The sentence to be recognized is abbcde.

NOTES.PMR-INSIGNIA.ORG

REDUCTION (LEFTMOST)

RIGHTMOST DERIVATION

abbcde (A b) S aABe aAbcde (A Abc) aAde aAde (B d) aAbcde aABe (S aABe) abbcde S The reductions trace out the right-most derivation in reverse. Handles: A handle of a string is a substring that matches the right side of a production, and whose reduction to the non-terminal on the left side of the production represents one step along the reverse of a rightmost derivation. Example: Consider the grammar: E E+E E E*E E (E) E id And the input string id1+id2*id3 The rightmost derivation is : E E+E E+E*E E+E*id3 E+id2*id3 id1+id2*id3 In the above derivation the underlined substrings are called handles. Handle pruning: A rightmost derivation in reverse can be obtained by handle pruning. (i.e.) if w is a sentence or string of the grammar at hand, then w = n, where n is the nth rightsentinel form of some rightmost derivation.

NOTES.PMR-INSIGNIA.ORG

Stack implementation of shift-reduce parsing : Stack 1. $ 2. $ id1 $E $ E+ $ E+id2 $ E+E $ E+E* $ E+E*id3 $ E+E*E $ E+E $E Input id1+id2*id3 $ +id2*id3 $ +id2*id3 $ id2*id3 $ *id3 $ *id3 $ id3 $ $ $ $ $ shift reduce by Eid shift shift reduce by Eid shift shift reduce by Eid reduce by E E *E reduce by E E+E accept Action

Actions in shift-reduce parser: shift The next input symbol is shifted onto the top of the stack. reduce The parser replaces the handle within a stack with a non-terminal. accept The parser announces successful completion of parsing. error The parser discovers that a syntax error has occurred and calls an error recovery routine. Conflicts in shift-reduce parsing: There are two conflicts that occur in shift shift-reduce parsing: 1. Shift-reduce conflict: The parser cannot decide whether to shift or to reduce. 2. Reduce-reduce conflict: The parser cannot decide which of several reductions to make. 1. Shift-reduce conflict: Example: Consider the grammar: EE+E | E*E | id and input id+id*id

NOTES.PMR-INSIGNIA.ORG

Stack $ E+E $E $ E* $ E*id $ E*E $E

Input *id $ *id $ id $ $ $

Action Reduce by EE+E Shift Shift Reduce by Eid Reduce by EE*E

Stack $E+E $E+E* $E+E*id $E+E*E $E+E $E

Input *id $ id $ $ $ $

Action Shift Shift Reduce by Eid Reduce by EE*E Reduce by EE*E

2. Reduce-reduce conflict: Consider the grammar: M R+R | R+c | R Rc and input c+c Stack $ $c $R $ R+ $ R+c $ R+R $M Input c+c $ +c $ +c $ c$ $ $ $ Action Shift Reduce by Rc Shift Shift Reduce by Rc Reduce by MR+R Stack $ $c $R $ R+ $ R+c $M Input c+c $ +c $ +c $ c$ $ $ Action Shift Reduce by Rc Shift Shift Reduce by MR+c

NOTES.PMR-INSIGNIA.ORG

Viable prefixes: is a viable prefix of the grammar if there is w such that w is a right sentinel form. The set of prefixes of right sentinel forms that can appear on the stack of a shift-reduce parser are called viable prefixes. The set of viable prefixes is a regular language.
OPERATOR-PRECEDENCE PARSING

An efficient way of constructing shift-reduce parser is called operator-precedence parsing. Operator precedence parser can be constructed from a grammar called Operator-grammar. These grammars have the property that no production on right side is or has two adjacent non terminals. Example: Consider the grammar: E EAE | (E) | -E | id A+|-|*|/| Since the right side EAE has three consecutive non-terminals, the grammar can be written as follows: E E+E | E-E | E*E | E/E | EE | -E | id Operator precedence relations: There are three disjoint precedence relations namely < . - less than = - equal to . > - greater than The relations give the following meaning: a < . b a yields precedence to b a = b a has the same precedence as b a . > b a takes precedence over b Rules for binary operations: 1. If operator 1 has higher precedence than operator 2, then make 1 . > 2 and 2 < . 1 2. If operators 1 and 2, are of equal precedence, then make 1 . > 2 and 2 . > 1 if operators are left associative 1 < . 2 and 2 < . 1 if right associative 3. Make the following for all operators : < . id , id . > < .( , ( < . ) .> , .> ) .> $ , $ < .

NOTES.PMR-INSIGNIA.ORG

Also make ( = ) , ( < . ( , ) . > ) , ( < . id , id . > ) , $ < . id , id . > $ , $ < . ( , ) . > $ Example: Operator-precedence relations for the grammar E E+E | E-E | E*E | E/E | EE | (E) | -E | id is given in the following table assuming 1. is of highest precedence and right-associative 2. * and / are of next higher precedence and left-associative, and 3. + and - are of lowest precedence and left-associative Note that the blanks in the table denote error entries. TABLE : Operator-precedence relations id * / . . . . > < < < <. . > <. <. <. <.
.

+ * / id ( ) $

+ > . >
. .

( <. <. <. <. <. <. <

) > . >
. .

$ > . >
. . . .

> . >
.

> . >
.

> . >
.

> . >
.

<. <. <. . > <.

<. <. <. <. <

> . >
.

> >

> . > <.

> . > <.

> . > <.

> . > <.

> . > =
.

> . >
.

>
.

>
.

>
.

>
.

>
.

>

>

<

<

<

<

<

Operator precedence parsing algorithm: Input : An input string w and a table of precedence relations. Output : If w is well formed, a skeletal parse tree ,with a placeholder non-terminal E labeling all interior nodes; otherwise, an error indication. Method : Initially the stack contains $ and the input buffer the string w $. To parse, we execute the following program : (1) Set ip to point to the first symbol of w$; (2) repeat forever (3) if $ is on top of the stack and ip points to $ then (4) return else begin (5) let a be the topmost terminal symbol on the stack and let b be the symbol pointed to by ip; (6) if a <. b or a = b then begin (7) push b onto the stack; (8) advance ip to the next input symbol; end;

NOTES.PMR-INSIGNIA.ORG

(9) (10) (11) (12)

else if a . > b then /*reduce*/ repeat pop the stack until the top stack terminal is related by <. to the terminal most recently popped (13) else error( ) end

Stack implementation of operator precedence parsing: Operator precedence parsing uses a stack and precedence relation table for its implementation of above algorithm. It is a shift-reduce parsing containing all four actions shift, reduce, accept and error. The initial configuration of an operator precedence parsing is STACK INPUT $ w$ where w is the input string to be parsed. Example: Consider the grammar E E+E | E-E | E*E | E/E | EE | (E) | id. Input string is id+id*id .The implementation is as follows: STACK $ $ id $ $+ $ +id $+ $+* $ + * id $+* $+ $ < > < < > < < > > > INPUT id+id*id $ +id*id $ +id*id $ id*id $ *id $ *id $ id $ $ $ $ $ COMMENT shift id pop the top of the stack id shift + shift id pop id shift * shift id pop id pop * pop + accept

Advantages of operator precedence parsing: 1. It is easy to implement. 2. Once an operator precedence relation is made between all pairs of terminals of a grammar , the grammar can be ignored. The grammar is not referred anymore during implementation. Disadvantages of operator precedence parsing: 1. It is hard to handle tokens like the minus sign (-) which has two different precedence. 2. Only a small class of grammar can be parsed using operator-precedence parser.

NOTES.PMR-INSIGNIA.ORG

LR PARSERS An efficient bottom-up syntax analysis technique that can be used to parse a large class of CFG is called LR(k) parsing. The L is for left-to-right scanning of the input, the R for constructing a rightmost derivation in reverse, and the k for the number of input symbols. When k is omitted, it is assumed to be 1. Advantages of LR parsing: It recognizes virtually all programming language constructs for which CFG can be written. It is an efficient non-backtracking shift-reduce parsing method. A grammar that can be parsed using LR method is a proper superset of a grammar that can be parsed with predictive parser. It detects a syntactic error as soon as possible. Drawbacks of LR method: It is too much of work to construct a LR parser by hand for a programming language grammar. A specialized tool, called a LR parser generator, is needed. Example: YACC. Types of LR parsing method: 1. SLR- Simple LR Easiest to implement, least powerful. 2. CLR- Canonical LR Most powerful, most expensive. 3. LALR- Look-Ahead LR Intermediate in size and cost between the other two methods. The LR parsing algorithm: The schematic form of an LR parser is as follows:

INPUT

a1

ai

an

Sm Xm Sm-1 Xm-1 S0 STACK

LR parsing program

OUTPUT

action

goto

NOTES.PMR-INSIGNIA.ORG

It consists of : an input, an output, a stack, a driver program, and a parsing table that has two parts (action and goto). The driver program is the same for all LR parser. The parsing program reads characters from an input buffer one at a time. The program uses a stack to store a string of the form s 0X1s1X2s2Xmsm, where sm is on top. Each Xi is a grammar symbol and each si is a state. The parsing table consists of two parts : action and goto functions.

Action : The parsing program determines sm, the state currently on top of stack, and a i, the current input symbol. It then consults action[sm,ai] in the action table which can have one of four values : 1. shift s, where s is a state, 2. reduce by a grammar production A , 3. accept, and 4. error. Goto : The function goto takes a state and grammar symbol as arguments and produces a state. LR Parsing algorithm: Input: An input string w and an LR parsing table with functions action and goto for grammar G. Output: If w is in L(G), a bottom-up-parse for w; otherwise, an error indication. Method: Initially, the parser has s0 on its stack, where s0 is the initial state, and w$ in the input buffer. The parser then executes the following program : set ip to point to the first input symbol of w$; repeat forever begin let s be the state on top of the stack and a the symbol pointed to by ip; if action[s, a] = shift s then begin push a then s on top of the stack; advance ip to the next input symbol end else if action[s, a] = reduce A then begin pop 2* | | symbols off the stack; let s be the state now on top of the stack; push A then goto[s, A] on top of the stack; output the production A end else if action[s, a] = accept then return else error( ) end

NOTES.PMR-INSIGNIA.ORG

CONSTRUCTING SLR(1) PARSING TABLE: To perform SLR parsing, take grammar as input and do the following: 1. Find LR(0) items. 2. Completing the closure. 3. Compute goto(I,X), where, I is set of items and X is grammar symbol. LR(0) items: An LR(0) item of a grammar G is a production of G with a dot at some position of the right side. For example, production A XYZ yields the four items : A . XYZ A X . YZ A XY . Z A XYZ . Closure operation: If I is a set of items for a grammar G, then closure(I) is the set of items constructed from I by the two rules: 1. Initially, every item in I is added to closure(I). 2. If A . B is in closure(I) and B is a production, then add the item B . to I , if it is not already there. We apply this rule until no more new items can be added to closure(I). Goto operation: Goto(I, X) is defined to be the closure of the set of all items [A X . ] such that [A . X] is in I. Steps to construct SLR parsing table for grammar G are: 1. Augment G and produce G 2. Construct the canonical collection of set of items C for G 3. Construct the parsing action function action and goto using the following algorithm that requires FOLLOW(A) for each non-terminal of grammar. Algorithm for construction of SLR parsing table: Input : An augmented grammar G Output : The SLR parsing table functions action and goto for G Method : 1. Construct C = {I0, I1, . In}, the collection of sets of LR(0) items for G. 2. State i is constructed from Ii.. The parsing functions for state i are determined as follows: (a) If [Aa] is in Ii and goto(Ii,a) = Ij, then set action[i,a] to shift j. Here a must be terminal. (b) If [A] is in I i , then set action[i,a] to reduce A for all a in FOLLOW(A). (c) If [SS.] is in Ii, then set action[i,$] to accept. If any conflicting actions are generated by the above rules, we say grammar is not SLR(1).

NOTES.PMR-INSIGNIA.ORG

3. 4. 5.

The goto transitions for state i are constructed for all non-terminals A using the rule: If goto(Ii,A) = Ij, then goto[i,A] = j. All entries not defined by rules (2) and (3) are made error The initial state of the parser is the one constructed from the set of items containing [S.S].

Example for SLR parsing: Construct SLR parsing for the following grammar : G:EE+T|T TT*F|F F (E) | id The given grammar is : G : E E + T ------ (1) E T ------ (2) T T * F ------ (3) TF ------ (4) F (E) ------ (5) F id ------ (6) Step 1 : Convert given grammar into augmented grammar. Augmented grammar : E E E E+T E T T T*F TF F (E) F id Step 2 : Find LR (0) items. I0 : E . E E .E+T E .T T .T*F T .F F . (E) F . id

GOTO ( I0 , E) I1 : E E . E E.+T

GOTO ( I4 , id ) I5 : F id .

NOTES.PMR-INSIGNIA.ORG

GOTO ( I0 , T) I2 : E T . T T.*F GOTO ( I0 , F) I3 : T F .

GOTO ( I6 , T ) I9 : E E + T . TT.*F GOTO ( I6 , F ) I3 : T F . GOTO ( I6 , ( ) I4 : F ( . E ) GOTO ( I6 , id) I5 : F id . GOTO ( I7 , F ) I10 : T T * F . GOTO ( I7 , ( ) I4 : F ( . E ) E .E+T E .T T .T*F T .F F . (E) F . id GOTO ( I7 , id ) I5 : F id . GOTO ( I8 , ) ) I11 : F ( E ) . GOTO ( I8 , + ) I6 : E E + . T T.T*F T.F F.(E) F . id GOTO ( I9 , *) I7 : T T * . F F.(E) F . id

GOTO ( I0 , ( ) I4 : F ( . E) E .E+T E .T T .T*F T .F F . (E) F . id GOTO ( I0 , id ) I5 : F id . GOTO ( I1 , + ) I6 : E E + . T T .T*F T .F F . (E) F . id GOTO ( I2 , * ) I7 : T T * . F F . (E) F . id GOTO ( I4 , E ) I8 : F ( E . ) EE.+T GOTO ( I4 , T) I2 : E T . TT.*F GOTO ( I4 , F) I3 : T F .

NOTES.PMR-INSIGNIA.ORG

GOTO ( I4 , ( ) I4 : F ( . E) E .E+T E .T T .T*F T .F F . (E) F id

FOLLOW (E) = { $ , ) , +) FOLLOW (T) = { $ , + , ) , * } FOOLOW (F) = { * , + , ) , $ } SLR parsing table: ACTION id I0 I1 I2 I3 I4 I5 I6 I7 I8 I9 I10 I11 s5 s5 s6 r1 r3 r5 s7 r3 r5 s5 r6 r6 s4 s4 s11 r1 r3 r5 r1 r3 r5 s5 s6 r2 r4 s7 r4 s4 r6 r6 9 3 10 r2 r4 + * ( s4 ACC r2 r4 8 2 3 ) $ E 1 GOTO T 2 F 3

Blank entries are error entries. Stack implementation: Check whether the input id + id * id is valid or not.

NOTES.PMR-INSIGNIA.ORG

STACK 0 0 id 5 0F3 0T2 0E1 0E1+6 0 E 1 + 6 id 5 0E1+6F3 0E1+6T9 0E1+6T9*7 0 E 1 + 6 T 9 * 7 id 5 0 E 1 + 6 T 9 * 7 F 10 0E1+6T9 0E1

INPUT id + id * id $ + id * id $ + id * id $ + id * id $ + id * id $ id * id $ * id $ * id $ * id $ id $ $ $ $ $

ACTION GOTO ( I0 , id ) = s5 ; shift GOTO ( I5 , + ) = r6 ; reduce by Fid GOTO ( I0 , F ) = 3 GOTO ( I3 , + ) = r4 ; reduce by T F GOTO ( I0 , T ) = 2 GOTO ( I2 , + ) = r2 ; reduce by E T GOTO ( I0 , E ) = 1 GOTO ( I1 , + ) = s6 ; shift GOTO ( I6 , id ) = s5 ; shift GOTO ( I5 , * ) = r6 ; reduce by F id GOTO ( I6 , F ) = 3 GOTO ( I3 , * ) = r4 ; reduce by T F GOTO ( I6 , T ) = 9 GOTO ( I9 , * ) = s7 ; shift GOTO ( I7 , id ) = s5 ; shift GOTO ( I5 , $ ) = r6 ; reduce by F id GOTO ( I7 , F ) = 10 GOTO ( I10 , $ ) = r3 ; reduce by T T * F GOTO ( I6 , T ) = 9 GOTO ( I9 , $ ) = r1 ; reduce by E E + T GOTO ( I0 , E ) = 1 GOTO ( I1 , $ ) = accept

TYPE CHECKING A compiler must check that the source program follows both syntactic and semantic conventions of the source language. This checking, called static checking, detects and reports programming errors. Some examples of static checks: 1. Type checks A compiler should report an error if an operator is applied to an incompatible operand. Example: If an array variable and function variable are added together.

NOTES.PMR-INSIGNIA.ORG

2. Flow-of-control checks Statements that cause flow of control to leave a construct must have some place to which to transfer the flow of control. Example: An error occurs when an enclosing statement, such as break, does not exist in switch statement. Position of type checker token stream parser syntax tree type checker syntax tree intermediate code generator intermediate representation

A type checker verifies that the type of a construct matches that expected by its context. For example : arithmetic operator mod in Pascal requires integer operands, so a type checker verifies that the operands of mod have type integer. Type information gathered by a type checker may be needed when code is generated.

TYPE SYSTEMS The design of a type checker for a language is based on information about the syntactic constructs in the language, the notion of types, and the rules for assigning types to language constructs. For example : if both operands of the arithmetic operators of +,- and * are of type integer, then the result is of type integer Type Expressions The type of a language construct will be denoted by a type expression. A type expression is either a basic type or is formed by applying an operator called a type constructor to other type expressions. The sets of basic types and constructors depend on the language to be checked.

The following are the definitions of type expressions: 1. Basic types such as boolean, char, integer, real are type expressions. A special basic type, type_error , will signal an error during type checking; void denoting the absence of a value allows statements to be checked. 2. 3. Since type expressions may be named, a type name is a type expression. A type constructor applied to type expressions is a type expression. Constructors include: Arrays : If T is a type expression then array (I,T) is a type expression denoting the type of an array with elements of type T and index set I. Products : If T1 and T2 are type expressions, then their Cartesian product T 1 X T2 is a type expression.

NOTES.PMR-INSIGNIA.ORG

Records : The difference between a record and a product is that the fields of a record have names. The record type constructor will be applied to a tuple formed from field names and field types. For example: type row = record address: integer; lexeme: array[1..15] of char end; var table: array[1...101] of row; declares the type name row representing the type expression record((address X integer) X (lexeme X array(1..15,char))) and the variable table to be an array of records of this type. Pointers : If T is a type expression, then pointer(T) is a type expression denoting the type pointer to an object of type T. For example, var p: row declares variable p to have type pointer(row). Functions : A function in programming languages maps a domain type D to a range type R. The type of such function is denoted by the type expression D R 4. Type expressions may contain variables whose values are type expressions. Tree representation for char x char pointer (integer) x char Type systems A type system is a collection of rules for assigning type expressions to the various parts of a program. A type checker implements a type system. It is specified in a syntax-directed manner. Different type systems may be used by different compilers or processors of the same language. Static and Dynamic Checking of Types Checking done by a compiler is said to be static, while checking done when the target program runs is termed dynamic. Any check can be done dynamically, if the target code carries the type of an element along with the value of that element. char pointer integer

NOTES.PMR-INSIGNIA.ORG

Sound type system A sound type system eliminates the need for dynamic checking for type errors because it allows us to determine statically that these errors cannot occur when the target program runs. That is, if a sound type system assigns a type other than type_error to a program part, then type errors cannot occur when the target code for the program part is run. Strongly typed language A language is strongly typed if its compiler can guarantee that the programs it accepts will execute without type errors. Error Recovery Since type checking has the potential for catching errors in program, it is desirable for type checker to recover from errors, so it can check the rest of the input. Error handling has to be designed into the type system right from the start; the type checking rules must be prepared to cope with errors.

SPECIFICATION OF A SIMPLE TYPE CHECKER Here, we specify a type checker for a simple language in which the type of each identifier must be declared before the identifier is used. The type checker is a translation scheme that synthesizes the type of each expression from the types of its subexpressions. The type checker can handle arrays, pointers, statements and functions. A Simple Language Consider the following grammar: P D;E D D ; D | id : T T char | integer | array [ num ] of T | T E literal | num | id | E mod E | E [ E ] | E Translation scheme: PD;E DD;D D id : T { addtype (id.entry , T.type) } T char { T.type : = char } T integer { T.type : = integer } T T1 { T.type : = pointer(T1.type) } T array [ num ] of T1 { T.type : = array ( 1 num.val , T1.type) } In the above language, There are two basic types : char and integer ; type_error is used to signal errors; the prefix operator builds a pointer type. Example , integer leads to the type expression pointer ( integer ).

NOTES.PMR-INSIGNIA.ORG

Type checking of expressions In the following rules, the attribute type for E gives the type expression assigned to the expression generated by E. 1. E literal { E.type : = char } E num { E.type : = integer } Here, constants represented by the tokens literal and num have type char and integer. 2. E id { E.type : = lookup ( id.entry ) } lookup ( e ) is used to fetch the type saved in the symbol table entry pointed to by e. 3. E E1 mod E2 { E.type : = if E1. type = integer and E2. type = integer then integer else type_error } The expression formed by applying the mod operator to two subexpressions of type integer has type integer; otherwise, its type is type_error. { E.type : = if E2.type = integer and E1.type = array(s,t) then t else type_error } In an array reference E1 [ E2 ] , the index expression E2 must have type integer. The result is the element type t obtained from the type array(s,t) of E1. { E.type : = if E1.type = pointer (t) then t else type_error }

4. E E1 [ E2 ]

5. E E1

The postfix operator yields the object pointed to by its operand. The type of E is the type t of the object pointed to by the pointer E. Type checking of statements Statements do not have values; hence the basic type void can be assigned to them. If an error is detected within a statement, then type_error is assigned. Translation scheme for checking the type of statements: 1. Assignment statement: S id : = E { S.type : = if id.type = E.type then void else type_error } 2. Conditional statement: S if E then S1 { S.type : = if E.type = boolean then S1.type else type_error } 3. While statement: S while E do S1 { S.type : = if E.type = boolean then S1.type else type_error }

NOTES.PMR-INSIGNIA.ORG

4. Sequence of statements: S S1 ; S2 { S.type : = if S1.type = void and S1.type = void then void else type_error } Type checking of functions The rule for checking the type of a function application is : E E 1 ( E2 ) { E.type : = if E2.type = s and E1.type = s t then t else type_error } SOURCE LANGUAGE ISSUES Procedures: A procedure definition is a declaration that associates an identifier with a statement. The identifier is the procedure name, and the statement is the procedure body. For example, the following is the definition of procedure named readarray : procedure readarray; var i : integer; begin for i : = 1 to 9 do read(a[i]) end; When a procedure name appears within an executable statement, the procedure is said to be called at that point. Activation trees: An activation tree is used to depict the way control enters and leaves activations. In an activation tree, 1. Each node represents an activation of a procedure. 2. The root represents the activation of the main program. 3. The node for a is the parent of the node for b if and only if control flows from activation a to b. 4. The node for a is to the left of the node for b if and only if the lifetime of a occurs before the lifetime of b. Control stack: A control stack is used to keep track of live procedure activations. The idea is to push the node for an activation onto the control stack as the activation begins and to pop the node when the activation ends. The contents of the control stack are related to paths to the root of the activation tree. When node n is at the top of control stack, the stack contains the nodes along the path from n to the root.

NOTES.PMR-INSIGNIA.ORG

The Scope of a Declaration: A declaration is a syntactic construct that associates information with a name. Declarations may be explicit, such as: var i : integer ; or they may be implicit. Example, any variable name starting with I is assumed to denote an integer. The portion of the program to which a declaration applies is called the scope of that declaration. Binding of names: Even if each name is declared once in a program, the same name may denote different data objects at run time. Data object corresponds to a storage location that holds values. The term environment refers to a function that maps a name to a storage location. The term state refers to a function that maps a storage location to the value held there. environment state

name

storage

value

When an environment associates storage location s with a name x, we say that x is bound to s. This association is referred to as a binding of x.

STORAGE ORGANISATION The executing target program runs in its own logical address space in which each program value has a location. The management and organization of this logical address space is shared between the complier, operating system and target machine. The operating system maps the logical address into physical addresses, which are usually spread throughout memory.

Typical subdivision of run-time memory: Code Static Data Stack free memory Heap

NOTES.PMR-INSIGNIA.ORG

Run-time storage comes in blocks, where a byte is the smallest unit of addressable memory. Four bytes form a machine word. Multibyte objects are stored in consecutive bytes and given the address of first byte. The storage layout for data objects is strongly influenced by the addressing constraints of the target machine. A character array of length 10 needs only enough bytes to hold 10 characters, a compiler may allocate 12 bytes to get alignment, leaving 2 bytes unused. This unused space due to alignment considerations is referred to as padding. The size of some program objects may be known at run time and may be placed in an area called static. The dynamic areas used to maximize the utilization of space at run time are stack and heap.

Activation records: Procedure calls and returns are usually managed by a run time stack called the control stack. Each live activation has an activation record on the control stack, with the root of the activation tree at the bottom, the latter activation has its record at the top of the stack. The contents of the activation record vary with the language being implemented. The diagram below shows the contents of activation record.

Temporary values such as those arising from the evaluation of expressions. Local data belonging to the procedure whose activation record this is. A saved machine status, with information about the state of the machine just before the call to procedures. An access link may be needed to locate data needed by the called procedure but found elsewhere. A control link pointing to the activation record of the caller.

NOTES.PMR-INSIGNIA.ORG

Space for the return value of the called functions, if any. Again, not all called procedures return a value, and if one does, we may prefer to place that value in a register for efficiency. The actual parameters used by the calling procedure. These are not placed in activation record but rather in registers, when possible, for greater efficiency.

STORAGE ALLOCATION STRATEGIES The different storage allocation strategies are : 1. Static allocation lays out storage for all data objects at compile time 2. Stack allocation manages the run-time storage as a stack. 3. Heap allocation allocates and deallocates storage as needed at run time from a data area known as heap.
STATIC ALLOCATION

In static allocation, names are bound to storage as the program is compiled, so there is no need for a run-time support package. Since the bindings do not change at run-time, everytime a procedure is activated, its names are bound to the same storage locations. Therefore values of local names are retained across activations of a procedure. That is, when control returns to a procedure the values of the locals are the same as they were when control left the last time. From the type of a name, the compiler decides the amount of storage for the name and decides where the activation records go. At compile time, we can fill in the addresses at which the target code can find the data it operates on.

STACK ALLOCATION OF SPACE

All compilers for languages that use procedures, functions or methods as units of userdefined actions manage at least part of their run-time memory as a stack. Each time a procedure is called , space for its local variables is pushed onto a stack, and when the procedure terminates, that space is popped off the stack.

Calling sequences: Procedures called are implemented in what is called as calling sequence, which consists of code that allocates an activation record on the stack and enters information into its fields. A return sequence is similar to code to restore the state of machine so the calling procedure can continue its execution after the call. The code in calling sequence is often divided between the calling procedure (caller) and the procedure it calls (callee). When designing calling sequences and the layout of activation records, the following principles are helpful: Values communicated between caller and callee are generally placed at the beginning of the callees activation record, so they are as close as possible to the callers activation record.

NOTES.PMR-INSIGNIA.ORG

Fixed length items are generally placed in the middle. Such items typically include the control link, the access link, and the machine status fields. Items whose size may not be known early enough are placed at the end of the activation record. The most common example is dynamically sized array, where the value of one of the callees parameters determines the length of the array. We must locate the top-of-stack pointer judiciously. A common approach is to have it point to the end of fixed-length fields in the activation record. Fixed-length data can then be accessed by fixed offsets, known to the intermediate-code generator, relative to the top-of-stack pointer. ... Parameters and returned values

callers activation record callers responsibility callees activation record callees responsibility

control link links and saved status temporaries and local data Parameters and returned values control link links and saved status temporaries and local data

top_sp

Division of tasks between caller and callee The calling sequence and its division between caller and callee are as follows. The caller evaluates the actual parameters. The caller stores a return address and the old value of top_sp into the callees activation record. The caller then increments the top_sp to the respective positions. The callee saves the register values and other status information. The callee initializes its local data and begins execution. A suitable, corresponding return sequence is: The callee places the return value next to the parameters. Using the information in the machine-status field, the callee restores top_sp and other registers, and then branches to the return address that the caller placed in the status field. Although top_sp has been decremented, the caller knows where the return value is, relative to the current value of top_sp; the caller therefore may use that value.

NOTES.PMR-INSIGNIA.ORG

Variable length data on stack: The run-time memory management system must deal frequently with the allocation of space for objects, the sizes of which are not known at the compile time, but which are local to a procedure and thus may be allocated on the stack. The reason to prefer placing objects on the stack is that we avoid the expense of garbage collecting their space. The same scheme works for objects of any type if they are local to the procedure called and have a size that depends on the parameters of the call.

. activation record for p

control link pointer to A pointer to B pointer to C array A array B array C

arrays of p

activation record for procedure q called by p arrays of q

control link

top_sp

top

Access to dynamically allocated arrays Procedure p has three local arrays, whose sizes cannot be determined at compile time. The storage for these arrays is not part of the activation record for p. Access to the data is through two pointers, top and top-sp. Here the top marks the actual top of stack; it points the position at which the next activation record will begin. The second top-sp is used to find local, fixed-length fields of the top activation record. The code to reposition top and top-sp can be generated at compile time, in terms of sizes that will become known at run time.

NOTES.PMR-INSIGNIA.ORG

HEAP ALLOCATION

Stack allocation strategy cannot be used if either of the following is possible : 1. The values of local names must be retained when an activation ends. 2. A called activation outlives the caller. Heap allocation parcels out pieces of contiguous storage, as needed for activation records or other objects. Pieces may be deallocated in any order, so over the time the heap will consist of alternate areas that are free and in use.

Position in the activation tree s r q ( 1 , 9)

Activation records in the heap

Remarks Retained activation record for r

s control link r control link

q(1,9) control link

The record for an activation of procedure r is retained when the activation ends. Therefore, the record for the new activation q(1 , 9) cannot follow that for s physically. If the retained activation record for r is deallocated, there will be free space in the heap between the activation records for s and q.

NOTES.PMR-INSIGNIA.ORG