Department of Information techonlogy
Compiler Design Lab
(22PCOIT20)
Implementation Of Lexical Analyser
Using Lex Tool
Presented by Under the guidence of:
SK.Ayesha Jabeen Mr.G.Shekar
22WJ1A1252 Assistant Professor
ABSTRACT
This project presents the implementation of a lexical analyzer using the Lex
tool. Lex is a powerful tool for generating lexical analyzers, which are
essential components of compilers and interpreters. The lexical analyzer is
designed to recognize and tokenize the input stream, identifying keywords,
identifiers, literals, and symbols. The implementation involves specifying the
lexical rules using regular expressions and generating the lexer code using the
Lex tool. The resulting lexical analyzer can be used in various applications,
including compiler design, text processing, and data analysis.
Introduction to Lexical Analyzers
A lexical analyzer, or lexer, is a component that converts a sequence of
characters into a sequence of tokens.It plays a crucial role in the compilation
process by breaking down source code into manageable pieces.Understanding
how to implement a lexical analyzer using the lex tool simplifies the process of
token generation.
A lexical analyzer, often called a lexer or scanner, is a crucial part of a
compiler that reads input text and breaks it down into meaningful units called
tokens. The Lex tool (also known as Flex) is a tool used to automatically
generate a lexical analyzer by specifying regular expressions for token patterns
and associating actions with these patterns.
Overview of the Lex Tool
• Lex is a popular tool that generates lexical analyzers from regular
expressions and associated actions.
• It takes a specification file as input and produces C code for the lexer,
which can be compiled and linked with a parser.
• The generated code is efficient and can be modified to handle complex
patterns and tokenization tasks.
Writing a Lex Specification File
• A lex specification file consists of
three main sections: definitions,
rules, and user code.
• The definitions section allows the
declaration of tokens, while the
rules section specifies patterns and
associated actions.
• User code can include additional
functions or libraries needed for the
lexer’s operation, enhancing its
functionality.
Compiling and Running the Lexer
• To compile the lexer, the lex tool is invoked, generating a C source file
from the specification provided.
• The generated C file is then compiled using a C compiler, typically
resulting in an executable file.
• Finally, the lexer can be run with input files to produce tokens, which can
be further processed by a parser.
Example of a Simple Lexer Implementation
• A simple example of a lexer might tokenize a basic arithmetic expression
with numbers and operators.
• The lex specification would define patterns for digits, operators, and
whitespace to ignore.
• Testing the lexer with various input strings will help ensure that it correctly
recognizes and processes tokens as intended
SOURCE CODE
%{
%}
identifier [a-z|A-Z]I[a-z|A-Z|0-9]*
%%
%%
#.*
int
(identifier}\(
\{
{identifier}
.|\n
{printf("\n%s is a preprocessor dir",yytext);}
{printf("'\n\t%s is a keyword",yytext);}
{printf("\n\nFUNCTION\n\t%s",yytext);}
{printf("\nBLOCK BEGINS");}
{printf("\nBLOCK ENDS");}
(printf("\n%s is an IDENTIFIER",yytext);}
int main(int argc,char **argv)
」
if(argc>1)
一
FILE *file;
file=fopen(argv[1],"r"):
if(Ifile)
{
printf("\n couldnot open %s\n",argv[1]);
exit(0);
}
yyin=file;
}
yylex();
printf("\n\n");
return 0;
int yywrap()
{
return 0;
Input (in.c)
#include<stdio.h>
main()
{
int a;
