Department of Computer and Science Engineering (CSE)

UNIVERSITY INSTITUTE OF
ENGINEERING
COMPUTER SCIENCE
ENGINEERING
Bachelor of Engineering
Theory of Computation (CST-353)

Topic: Introduction
DISCOVER . LEARN .
University Institute of Engineering (UIE) EMPOWER
 Department of Computer and Science Engineering (CSE)

Learning Objectives & Outcomes

Objective:
• To understand the basics to automata.

Outcome:
• Student will understand the
 Basics of automata

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

What is Automata Theory?

• Study of abstract computing devices, or

“machines”
• Automaton = an abstract computing device
• A fundamental question in computer science:
– Find out what different models of machines can do and
cannot do
– The theory of computation
• Computability vs. Complexity

3
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

(A pioneer of automata theory)

Alan Turing (1912-1954)

• Father of Modern Computer

Science

• English mathematician

• Studied abstract machines called

Turing machines even before
computers existed

• Heard of the Turing test?

4
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Theory of Computation: A Historical Perspective

1930s • Alan Turing studies Turing machines

• Decidability
• Halting problem

1940-1950s • “Finite automata” machines studied

• Noam Chomsky proposes the
“Chomsky Hierarchy” for formal
languages
1969 Cook introduces “intractable” problems
or “NP-Hard” problems
Modern computer science: compilers,
1970-
computational & complexity theory evolve
5
University Institute of Engineering (UIE)
 Languages & Grammars
• Languages: “A language is a
Or “words” collection of sentences of finite
length all constructed from a
finite alphabet of symbols”
• Grammars: “A grammar can
be regarded as a device that
enumerates the sentences of a
language” - nothing more,
nothing less

• N. Chomsky, Information and

Control, Vol 2, 1959

Image source: Nowak et al. Nature, vol 417, 2002

University Institute of Engineering (UIE) 6
 The Chomsky Hierachy
• A containment hierarchy of classes of formal languages

Regular Context- Recursively-

(DFA) Context-
free enumerable
sensitive
(PDA) (TM)
(LBA)

University Institute of Engineering (UIE) 7

 Finite Automata
1. Finite automata are used to recognize patterns.

2. It takes the string of symbol as input and changes its

state accordingly. When the desired symbol is found,
then the transition occurs.

3. At the time of transition, the automata can either move

to the next state or stay in the same state.

4. Finite automata have two states, Accept

state or Reject state. When the input string is
processed successfully, and the automata reached its
final state, then it will accept.

University Institute of Engineering (UIE) 8

 Department of Computer and Science Engineering (CSE)

Finite Automata : Examples

• On/Off switch action

state

• Modeling recognition of the word “then”

Start state Transition Intermediate Final state

state
9
University Institute of Engineering (UIE)
 Formal Definition of FA
A finite automaton is a collection of 5-tuple (Q, ∑, δ, q0, F),
where:

Q: finite set of states

∑: finite set of the input symbol

q0: initial state

F: final state

δ: Transition function

University Institute of Engineering (UIE) 10

 Finite Automata Model:

Finite automata can be represented by input tape and finite

control.

Input tape: It is a linear tape having some number of cells.

Each input symbol is placed in each cell.

Finite control: The finite control decides the next state on

receiving particular input from input tape. The tape reader
reads the cells one by one from left to right, and at a time only
one input symbol is read.

University Institute of Engineering (UIE) 11

University Institute of Engineering (UIE) 12
 Types of Automata:

There are two types of finite automata:

DFA(deterministic finite automata)

NFA(non-deterministic finite automata)

University Institute of Engineering (UIE) 13

Some important points about DFA and NFA:

Every DFA is NFA, but NFA is not DFA.

There can be multiple final states in both NFA and DFA.

DFA is used in Lexical Analysis in Compiler.

NFA is more of a theoretical concept.

University Institute of Engineering (UIE) 14

 Department of Computer and Science Engineering (CSE)

Alphabet

An alphabet is a finite, non-empty set of symbols

• We use the symbol ∑ (sigma) to denote an
alphabet
• Examples:
– Binary: ∑ = {0,1}
– All lower case letters: ∑ = {a,b,c,..z}
– Alphanumeric: ∑ = {a-z, A-Z, 0-9}
– DNA molecule letters: ∑ = {a,c,g,t}
– …

15
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Powers of an alphabet
Let ∑ be an alphabet.

– ∑k = the set of all strings of length k

– ∑* = ∑0 U ∑1 U ∑2 U …

– ∑+ = ∑1 U ∑2 U ∑3 U …

16
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Strings

A string or word is a finite sequence of symbols

chosen from ∑
• Empty string is  (or “epsilon”)

• Length of a string w, denoted by “|w|”, is equal to

the number of (non- ) characters in the string
– E.g., x = 010100 |x| = 6
– x = 01  0  1  00  |x| = ?

– xy = concatentation of two strings x and y

17
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Summary
• Automata theory & a historical perspective
• Chomsky hierarchy
• Finite automata

18
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

FAQ :

1. What are types of finite automata.

2. Explain model of finite automata.

3. Explain tuples of finite automata.

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

References :
• Martin J.C., “Introduction to Languages and Theory of
Computation”, Tata McGraw-Hill Publising Company
Limited, 3rd Edition.
• https://youtu.be/S3cOulqSAmU
• Https://en.wikipedia.org/wiki/Finite-state_machine
• https://www.safaribooksonline.com
• https://nptel.ac.in/courses/106/103/106103070/

University Institute of Engineering (UIE)

THANK YOU

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

UNIVERSITY INSTITUTE OF
ENGINEERING
COMPUTER SCIENCE
ENGINEERING
Bachelor of Engineering
Theory of Computation (CST-353)

Topic: Introduction
DISCOVER . LEARN .
University Institute of Engineering (UIE) EMPOWER
 Department of Computer and Science Engineering (CSE)

Learning Objectives & Outcomes

Objective:
• To understand the concept of Formal Language,
Operations and examples of formal languages.

Outcome:
• Student will understand the
 Formal Language
 Operation on formal language

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Languages
L is a said to be a language over alphabet ∑, only if L  ∑*
 this is because ∑* is the set of all strings (of all possible length
including 0) over the given alphabet ∑
Examples:
1. Let L be the language of all strings consisting of n 0’s followed by n
1’s:
L = {,01,0011,000111,…}
2. Let L be the language of all strings of with equal number of 0’s and
1’s:
L = {,01,10,0011,1100,0101,1010,1001,…}

Definition: Ø denotes the Empty language

• Let L = {}; Is L=Ø? NO

3
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Language:
A language is a collection of appropriate string. A language
which is formed over Σ can be Finite or Infinite.

Example: 1
• L1 = {Set of string of length 2}
= {aa, bb, ba, bb} Finite Language

Example: 2
• L2 = {Set of all strings starts with 'a'}
= {a, aa, aaa, abb, abbb, ababb} Infinite Language

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Finite Automata

• Some Applications
– Software for designing and checking the behavior of
digital circuits
– Lexical analyzer of a typical compiler
– Software for scanning large bodies of text (e.g., web
pages) for pattern finding
– Software for verifying systems of all types that have a
finite number of states (e.g., stock market transaction,
communication/network protocol)

5
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Set operations on languages :

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Example of Formal Language:

• A language is a collection of appropriate string. A language

which is formed over Σ can be Finite or Infinite.

Example: 1
• L1 = {Set of string of length 2}
= {aa, bb, ba, bb} Finite Language

Example 2 :
• Example of Finite Language: L1 = { set of string of 2 } L1 = { xy,
yx, xx, yy }

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Example: 3
• L2 = {Set of all strings starts with 'a'}

= {a, aa, aaa, abb, abbb, ababb} Infinite Language

Example 4 :
• Example of Infinite Language: L1 = { set of all strings
starts with 'b' } L1 = { babb, baa, ba, bbb, baab, ....... }

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Summary
• Operations on Formal language
• Alphabets
• strings
• languages

10
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

FAQ :

1. Construct the powerset for the following sets.

• {a, b}
• {0, 1}∪{1, 2}
• {z}
• {0, 1, 2, 3, 4}∩{1, 3, 5, a}
• {0, 1, 2, 3}−{1, 3, 5, a}
• ∅ (the empty set)
2. Prove that 12 +22+32+……+ n2= ∑n2 =
(n(n+1)(2n+1))/6.

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

References :
• Martin J.C., “Introduction to Languages and Theory of
Computation”, Tata McGraw-Hill Publising Company
Limited, 3rd Edition.
• https://youtu.be/S3cOulqSAmU
• Https://en.wikipedia.org/wiki/Finite-state_machine
• https://www.safaribooksonline.com
• https://nptel.ac.in/courses/106/103/106103070/

University Institute of Engineering (UIE)

THANK YOU

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

UNIVERSITY INSTITUTE OF
ENGINEERING
COMPUTER SCIENCE
ENGINEERING
Bachelor of Engineering
Theory of Computation (CST-353)

Topic: Equivalence of Finite

DISCOVER . LEARN .
Automata
University Institute of Engineering (UIE) EMPOWER
 Department of Computer and Science Engineering (CSE)

Learning Objectives & Outcomes

Objective:
• To understand the concept of Finite Automata, and
equivalence of Finite Automata.

Outcome:
• Student will understand the
 Equivalence of DFA and NDFA
 NFA with null transition

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Equivalence of DFA & NFA

• Theorem:
– A language L is accepted by a DFA if and only if it is
Should be accepted by an NFA.
true for
any L • Proof:
1. If part:
• Prove by showing every NFA can be converted to an
equivalent DFA (in the next few slides…)

2. Only-if part is trivial:

• Every DFA is a special case of an NFA where each state has
exactly one transition for every input symbol. Therefore, if L is
accepted by a DFA, it is accepted by a corresponding NFA.

3
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Proof for the if-part

• If-part: A language L is accepted by a DFA if it is

accepted by an NFA
• rephrasing…
• Given any NFA N, we can construct a DFA D such
that L(N)=L(D)
• How to convert an NFA into a DFA?
– Observation: In an NFA, each transition maps to a subset
of states
– Idea: Represent:
each “subset of NFA_states”  a single “DFA_state”

4
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

NFA to DFA by subset construction

• Let N = {QN,∑,δN,q0,FN}
• Goal: Build D={QD,∑,δD,{q0},FD} s.t.
L(D)=L(N)
• Construction:
1. QD= all subsets of QN (i.e., power set)
2. FD=set of subsets S of QN s.t. S∩FN≠Φ
3. δD: for each subset S of QN and for each input
symbol a in ∑:
• δD(S,a) = U δN(p,a)
p in s

5
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

NFA to DFA construction: Example

• L = {w | w ends in 01} Idea: To avoid
enumerating all of
1 0 power set, do “lazy
creation of states”
NFA: DFA: 0 1
0,1 {q0} {q0,q1} {q0,q2}
0
0 1 1
q0 q1 q2

δD 0 1 δD 0 1
δN 0 1
Ø Ø Ø {q0} {q0,q1} {q0}
q0 {q0,q1} {q0} {q0} {q0,q1} {q0} {q0,q1} {q0,q1} {q0,q2}
q1 Ø {q2} {q1} Ø {q2} *{q0,q2} {q0,q1} {q0}
*q2 Ø Ø *{q2} Ø Ø
{q0,q1} {q0,q1} {q0,q2} 0. Enumerate all possible subsets
*{q0,q2} {q0,q1} {q0} 1. Determine transitions
*{q1,q2} Ø {q2} 2. Retain only those states
reachable from {q0}
*{q0,q1,q2} {q0,q1} {q0,q2}

6
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

NFA to DFA: Repeating the example

using LAZY CREATION
• L = {w | w ends in 01}
1 0
NFA: DFA: 0 1
0,1 {q0} {q0,q1} {q0,q2}
0
0 1 1
q0 q1 q2

δN 0 1 δD 0 1
q0 {q0,q1} {q0} {q0} {q0,q1} {q0}
q1 Ø {q2} {q0,q1} {q0,q1} {q0,q2}
*q2 Ø Ø *{q0,q2} {q0,q1} {q0}

Main Idea: Introduce

states as you go(on a
need basis)

7
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Correctness of subset construction

Theorem: If D is the DFA constructed from NFA N by subset
construction, then L(D)=L(N)
• Proof:
– Show that δD({q0},w) ≡ δN(q0,w} , for all w
– Using induction on w’s length:
• Let w = xa
• δD({q0},xa) ≡ δD( δN(q0,x}, a ) ≡ δN(q0,w}

8
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

A bad case where #states(DFA)>>#states(NFA)

• L = {w | w is a binary string s.t., the kth symbol

from its end is a 1}

– NFA has k+1 states

– But an equivalent DFA needs to have at least 2k states

(Pigeon hole principle)

– m holes and >m pigeons
• => at least one hole has to contain two or more pigeons

9
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Applications

• Text indexing
– inverted indexing
– For each unique word in the database, store all locations
that contain it using an NFA or a DFA
• Find pattern P in text T
– Example: Google querying
• Extensions of this idea:
– PATRICIA tree, suffix tree

10
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

A few subtle properties of DFAs and NFAs

• The machine never really terminates.
– It is always waiting for the next input symbol or making transitions.
• The machine decides when to consume the next symbol from the
input and when to ignore it.
– (but the machine can never skip a symbol)
• => A transition can happen even without really consuming an input
symbol (think of consuming  as a free token) – if this happens, then
it becomes an -NFA (see next few slides).
• A single transition cannot consume more than one (non-) symbol.

11
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

FA with -Transitions

• We can allow explicit -transitions in finite

automata
– i.e., a transition from one state to another state without
consuming any additional input symbol
– Makes it easier sometimes to construct NFAs

Definition:  -NFAs are those NFAs with at least

one explicit -transition defined.
•  -NFAs have one more column in their transition
table

12
University Institute of Engineering (UIE)
 Example of an -NFA

L = {w | w is empty, or if non-empty will end in 01}

0,1

0 1
• -closure of a state q,
q0 q1 q2 ECLOSE(q), is the set of

all states (including
start q’0 itself) that can be
δE 0 1
 reached from q by
*q’0 Ø Ø {q’0,q0} repeatedly making an
q0 {q0,q1} {q0} {q0} arbitrary number of -
q1 Ø {q2} {q1} transitions.
*q2 Ø Ø {q2}

University Institute of Engineering (UIE) 13

 Example of an -NFA

To simulate any transition:

Step 1) Go to all immediate destination states.
Step 2) From there go to all their -closure states as well.

L = {w | w is empty, or if non-empty will end in 01}

0,1

0 1
Simulate for w=101:
q0 q1 q2
 q0’
 
start q’0
q0’ q0
δE 1 1
0 1
 Ø q0
*q’0 Ø Ø {q’0,q0} x 0
ECLOSE(q’0)
q0 {q0,q1} {q0} {q0} q1
q1 Ø {q2} {q1} ECLOSE(q0) 1
*q2 Ø Ø {q2} q2

University Institute of Engineering (UIE) 14

 Example of another -NFA
To simulate any transition:
Step 1) Go to all immediate destination states.
Step 2) From there go to all their -closure states as well.

0,1 Simulate for w=101: ?

0 1
q0 q1 q2
  1

start q’0 q3

δE 0 1

*q’0 Ø Ø {q’0,q0,q3}
q0 {q0,q1} {q0} {q0,q3}
q1 Ø {q2} {q1}
*q2 Ø Ø {q2}
q3 Ø {q2} {q3}

15
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Equivalency of DFA, NFA, -NFA

• Theorem: A language L is accepted by some -NFA if
and only if L is accepted by some DFA

• Implication:
– DFA ≡ NFA ≡ -NFA
– (all accept Regular Languages)

16
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Eliminating -transitions
Let E = {QE,∑,δE,q0,FE} be an -NFA
Goal: To build DFA D={QD,∑,δD,{qD},FD} s.t. L(D)=L(E)
Construction:
1. QD= all reachable subsets of QE factoring in -closures
2. qD = ECLOSE(q0)
3. FD=subsets S in QD s.t. S∩FE≠Φ
4. δD: for each subset S of QE and for each input symbol a∑:
• Let R= U δE(p,a) // go to destination states

• δD(S,a) = U ECLOSE(r) // from there, take a union

p in s of all their -closures

r in R

17
University Institute of Engineering (UIE)
 Example: -NFA  DFA

L = {w | w is empty, or if non-empty will end in 01}

0,1

0 1
q0 q1 q2

start q’0

δE 0 1
 δD 0 1
*q’0 Ø Ø {q’0,q0}
q0 {q0,q1} {q0} {q0} *{q’0,q0}

q1 Ø {q2} {q1} …

*q2 Ø Ø {q2}

University Institute of Engineering (UIE) 18

 Department of Computer and Science Engineering (CSE)

Summary
• Equivalency of DFA & NFA
• Removal of redundant states and including dead states
• -transitions in NFA
• Pigeon hole principles
• Text searching applications

19
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

References
• Https://en.wikipedia.org/wiki/Finite-state_machine
• https://www.safaribooksonline.com
• http://studentsfocus.com/
• http://www.francisxavier.ac.in/

University Institute of Engineering (UIE)

THANK YOU

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

UNIVERSITY INSTITUTE OF
ENGINEERING
COMPUTER SCIENCE
ENGINEERING
Bachelor of Engineering
Theory of Computation (CST-353)

Topic: Arden’s Theorem

DISCOVER . LEARN .
University Institute of Engineering (UIE) EMPOWER
 Department of Computer and Science Engineering (CSE)

Learning Objectives & Outcomes

Objective:
• To understand the concept of Arden’s Theorem.

Outcome:
• Student will understand the
 Arden’s Theorem

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Arden’s Theorem

In order to find out a regular expression of a Finite

Automaton, we use Arden’s Theorem along with the
properties of regular expressions.
Statement −
• Let P and Q be two regular expressions.
• If P does not contain null string, then R = Q + RP has a
unique solution that is R = QP*

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

PROOF

R = Q + (Q + RP)P [After putting the value R = Q + RP]

= Q + QP + RPP
When we put the value of R recursively again and again, we get the
following equation −
R = Q + QP + QP2 + QP3…..
R = Q (ε + P + P2 + P3 + …. )
R = QP* [As P* represents (ε + P + P2 + P3 + ….) ]
Hence, proved.

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Construct a regular expression corresponding to the

automata given below −

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

• Solution −
• Here the initial state is q1 and the final state is q2
• Now we write down the equations −
• q1 = q10 + ε
• q2 = q11 + q20
• q3 = q21 + q30 + q31
• Now, we will solve these three equations −
• q1 = ε0* [As, εR = R]
• So, q1 = 0*
• q2 = 0*1 + q20
• So, q2 = 0*1(0)* [By Arden’s theorem]
• Hence, the regular expression is 0*10*.

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Construct a regular expression corresponding to the

automata given below −

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

• Solution −
• Here the initial state and final state is q1.
• The equations for the three states q1, q2, and q3 are as follows
−
• q1 = q1a + q3a + ε (ε move is because q1 is the initial state0
• q2 = q1b + q2b + q3b
• q3 = q2a
• Now, we will solve these three equations −
• q2 = q1b + q2b + q3b
• = q1b + q2b + (q2a)b (Substituting value of q3)
• = q1b + q2(b + ab)
• = q1b (b + ab)* (Applying Arden’s Theorem)
• q1 = q1a + q3a + ε

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

• q1a + q2aa + ε (Substituting value of q3)

• = q1a + q1b(b + ab*)aa + ε (Substituting value of q2)
• = q1(a + b(b + ab)*aa) + ε
• = ε (a+ b(b + ab)*aa)*
• = (a + b(b + ab)*aa)*
• Hence, the regular expression is (a + b(b + ab)*aa)*.

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Find regular expression for the following DFA using

Arden’s Theorem-

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

• Solution-
•
• Step-01:
•
• Form a equation for each state-
• A = ∈ + B.1 ……(1)
• B = A.0 ……(2)
•
• Step-02:
•
• Bring final state in the form R = Q + RP.
•
• Using (1) in (2), we get-
• B = (∈ + B.1).0
• B = ∈.0 + B.1.0
• B = 0 + B.(1.0) ……(3)
•
• Using Arden’s Theorem in (3), we get-
• B = 0.(1.0)*
•
• Thus, Regular Expression for the given DFA = 0(10)*

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Find regular expression for the following DFA using

Arden’s Theorem-

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)
• Solution-
• Step-01:
• Form a equation for each state-
• q1 = ∈ ……(1)
• q2 = q1.a ……(2)
• q3 = q1.b + q2.a + q3.a …….(3)
• Step-02:
• Bring final state in the form R = Q + RP.
• Using (1) in (2), we get-
• q2 = ∈.a
• q2 = a …….(4)
• Using (1) and (4) in (3), we get-
• q3 = q1.b + q2.a + q3.a
• q3 = ∈.b + a.a + q3.a
• q3 = (b + a.a) + q3.a …….(5)
• Using Arden’s Theorem in (5), we get-
• q3 = (b + a.a)a*

• Thus, Regular Expression for the given DFA = (b + aa)a*

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Summary

• Arden’s Theorem
– Conversion of FA to R.E

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

References

• K.L.P MISHRA,“Introduction to computer Theory” .

• https://www.tutorialspoint.com/automata_theory/arde
ns_theorem.htm

University Institute of Engineering (UIE)

THANK YOU

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

UNIVERSITY INSTITUTE OF
ENGINEERING
COMPUTER SCIENCE
ENGINEERING
Bachelor of Engineering
Theory of Computation (CST-353)

Topic: Regular Expression

DISCOVER . LEARN .
University Institute of Engineering (UIE) EMPOWER
 Department of Computer and Science Engineering (CSE)

Learning Objectives & Outcomes

Objective:
• To understand the concept of Regular expression.

Outcome:
• Student will understand the
 Regular Language
 Regular expression
 Arden’s Theorem

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Regular Languages

Definition
• We define regular languages as the smallest class of
languages which contains all finite languages and closed
with respect to union, concatenation and Kleene closure.
• Every regular expression denotes a regular language.
Example
• (a*b) is a regular expression that denotes the set of
strings formed by a sequence, also empty, of a's followed
by a b.

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Regular Expressions vs. Finite

•
Automata
Offers a declarative way to express the pattern of any
string we want to accept
– E.g., 01*+ 10*

• Automata => more machine-like

< input: string , output: [accept/reject] >
• Regular expressions => more program syntax-like

• Unix environments heavily use regular expressions

– E.g., bash shell, grep, vi & other editors, sed
• Perl scripting – good for string processing
• Lexical analyzers such as Lex or Flex

4
University Institute of Engineering (UIE)
 Regular Expressions

Regular = Finite Automata

expressions (DFA, NFA, -NFA)
Syntactical
expressions Automata/machines

Regular
Languages

Formal language
classes

University Institute of Engineering (UIE) 5

 Department of Computer and Science Engineering (CSE)

Language Operators

• Union of two languages:

– L U M = all strings that are either in L or M
– Note: A union of two languages produces a third
language

• Concatenation of two languages:

– L . M = all strings that are of the form xy
s.t., x  L and y  M
– The dot operator is usually omitted
• i.e., LM is same as L.M

6
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Kleene Closure (the * operator)

• Kleene Closure of a given language L:
– L0= {}
– L1= {w | for some w  L}
– L2= { w1w2 | w1  L, w2  L (duplicates allowed)}
– Li= { w1w2…wi | all w’s chosen are  L (duplicates allowed)}
– (Note: the choice of each wi is independent)
– L* = Ui≥0 Li (arbitrary number of concatenations)
Example:
• Let L = { 1, 00}
– L0= {}
– L1= {1,00}
– L2= {11,100,001,0000}
– L3= {111,1100,1001,10000,000000,00001,00100,0011}
– L* = L0 U L1 U L2 U …

7
University Institute of Engineering (UIE)
 Kleene Closure (special notes)

• L* is an infinite set iff |L|≥1 and L≠{}

• If L={}, then L* = {}
• If L = Φ, then L* = {}

Σ* denotes the set of all words over an

alphabet Σ
– Therefore, an abbreviated way of saying there is
an arbitrary language L over an alphabet Σ is:
• L  Σ*

University Institute of Engineering (UIE) 8

 Department of Computer and Science Engineering (CSE)

Building Regular Expressions

• Let E be a regular expression and the language

represented by E is L(E)
• Then:
– (E) = E
– L(E + F) = L(E) U L(F)
– L(E F) = L(E) L(F)
– L(E*) = (L(E))*

9
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Example: how to use these regular

expression properties and language
operators?
• L = { w | w is a binary string which does not contain two consecutive 0s or two
consecutive 1s anywhere)
– E.g., w = 01010101 is in L, while w = 10010 is not in L
• Goal: Build a regular expression for L
• Four cases for w:
– Case A: w starts with 0 and |w| is even
– Case B: w starts with 1 and |w| is even
– Case C: w starts with 0 and |w| is odd
– Case D: w starts with 1 and |w| is odd
• Regular expression for the four cases:
– Case A: (01)*
– Case B: (10)*
– Case C: 0(10)*
– Case D: 1(01)*
• Since L is the union of all 4 cases:
– Reg Exp for L = (01)* + (10)* + 0(10)* + 1(01)*
• If we introduce  then the regular expression can be simplified to:
– Reg Exp for L = ( +1)(01)*( +0)

10
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Precedence of Operators

• Highest to lowest
– * operator (star)

– . (concatenation)
– + operator

• Example:
– 01* + 1 =( 0 . ((1)*) ) + 1

11
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Finite Automata (FA) & Regular

Expressions (Reg Ex)
• To show that they are interchangeable, consider
the following theorems:
– Theorem 1: For every DFA A there exists a regular
Proofs expression R such that L(R)=L(A)
in the book – Theorem 2: For every regular expression R there exists
an  -NFA E such that L(E)=L(R)

 -NFA NFA
Theorem 2 Kleene Theorem

Reg Ex DFA

12
University Institute of Engineering (UIE)
 DFA to RE construction
DFA Reg Ex
Theorem 1
Informally, trace all distinct paths (traversing cycles only once)
from the start state to each of the final states
and enumerate all the expressions along the way

Example: 1 0 0,1

q0 0 q1 1 q2

(1*) 0 (0*) 1 (0 + 1)*

1* 00* 1 (0+1)*

1*00*1(0+1)*
University Institute of Engineering (UIE) 13
 RE to -NFA construction
Reg Ex  -NFA
Theorem 2

Example: (0+1)*01(0+1)*

(0+1)* 01 (0+1)*

 

0
 
0

 0 1

 1
  1

 

University Institute of Engineering (UIE) 14

 Department of Computer and Science Engineering (CSE)

Algebraic Laws of Regular Expressions

• Commutative:
– E+F = F+E
• Associative:
– (E+F)+G = E+(F+G)
– (EF)G = E(FG)
• Identity:
– E+Φ = E
– E=E=E
• Annihilator:
– ΦE = EΦ = Φ

15
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Algebraic Laws…

• Distributive:
– E(F+G) = EF + EG
– (F+G)E = FE+GE
• Idempotent: E + E = E
• Involving Kleene closures:
– (E*)* = E*
– Φ* =
– * =
– E+ =EE*
– E? =  +E

16
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

True or False?

Let R and S be two regular expressions. Then:

1. ((R*)*)* = R* ?

2. (R+S)* = R* + S*?

3. (RS + R)* RS = (RR*S)*?

17
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Summary
• Regular expressions
• DFA to regular expression conversion
• Regular expression to -NFA conversion
• Algebraic laws of regular expressions.

18
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

FAQ

1. Design FA which accepts even number of 0’s and even no

of 1’s.
2. Prove that for every integer n≥0 the number 42n+1+ 3
n+2 is multiple of 13
3. Prove that 6n ≡ 0 (mod 9) for all integers n ≥ 2.

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

References

• K.L.P MISHRA,“Introduction to computer Theory” .

• https://youtu.be/yKOmggxrYmY
• https://youtu.be/cFd3pz5H-TA
• https://www.tutorialspoint.com/automata_theory/arde
ns_theorem.htm

University Institute of Engineering (UIE)

THANK YOU

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

UNIVERSITY INSTITUTE OF
ENGINEERING
COMPUTER SCIENCE
ENGINEERING
Bachelor of Engineering
Theory of Computation (CST-353)

Topic: Minimization of DFA

DISCOVER . LEARN .
EMPOWER
 Department of Computer and Science Engineering (CSE)

Learning Objectives & Outcomes

Objective:
• To understand the concept Equivalence and Minimization
of DFA.

Outcome:
• Student will understand the
 Minimization of DFA
 Myhill Nerode Theorem
 Applications of interest

• Comparing two DFAs:

– L(DFA1) == L(DFA2)?

• How to minimize a DFA?

1. Remove unreachable states
2. Identify & condense equivalent states into one

3
 When to call two states in a DFA
“equivalent”?
Two states p and q are said to be equivalent iff:
i) Any string w accepted by starting at p is also accepted by starting
at q;

p
w
AND
q
i) Any string w rejected by starting at p is also rejected by starting at
q.
p
w
q
 p≡q
4
 Computing equivalent states in a DFA
(Myhill- Nerode Theorem )

A =
0 1
B = =
0 1 0
A C E G C x x =
1 0 1
0 1 D x x x =
1 1 0 E x x x x =
B D F H
Pass #0 1 0 F x x x x x =
1. Mark accepting states ≠ non-accepting states
0
Pass #1 G x x x = x x =
1. Compare every pair of states
2. Distinguish by one symbol transition H x x = x x x x =
3. Mark = or ≠ or blank(tbd) A B C D E F G H
Pass #2
1. Compare every pair of states
2. Distinguish by up to two symbol transitions (until different or same or tbd)
….
(keep repeating until table complete)

5
Table Filling Algorithm - step by step

A =
0 1
B =
0 1 0
A C E G C =
1 0 1
0 1 D =
1 1 0 E =
B D F H
1 0 F =
0
G =
H =
A B C D E F G H

6
 Table Filling Algorithm - step by step

A =
0 1
B =
0 1 0
A C E G C =
1 0 1
0 1 D =
1 1 0 E X X X X =
B D F H
1 0 F X =
0
G X =
1. Mark X between accepting vs. non-accepting state H X =
A B C D E F G H

7
 Table Filling Algorithm - step by step

A =
0 1
B =
0 1 0
A C E G C X =
1 0 1
0 1 D X =
1 1 0 E X X X X =
B D F H
1 0 F X =
0
G X X =
1. Mark X between accepting vs. non-accepting state H X X =
2. Look 1- hop away for distinguishing states or strings A B C D E F G H

8
 Table Filling Algorithm - step by step

A =
0 1
B =
0 1 0
A C E G C X X =
1 0 1
0 1 D X X =
1 1 0 E X X X X =
B D F H
1 0 F X =
0
G X X X =
1. Mark X between accepting vs. non-accepting state H X X X =
2. Look 1- hop away for distinguishing states or strings A B C D E F G H

9
 Table Filling Algorithm - step by step

A =
0 1
B =
0 1 0
A C E G C X X =
1 0 1
0 1 D X X X =
1 1 0 E X X X X =
B D F H
1 0 F X X =
0
G X X X X =
1. Mark X between accepting vs. non-accepting state H X X = X =
2. Look 1- hop away for distinguishing states or strings A B C D E F G H

10
 Table Filling Algorithm - step by step

A =
0 1
B =
0 1 0
A C E G C X X =
1 0 1
0 1 D X X X =
1 1 0 E X X X X =
B D F H
1 0 F X X X =
0
G X X X = X =
1. Mark X between accepting vs. non-accepting state H X X = X X =
2. Look 1- hop away for distinguishing states or strings A B C D E F G H

11
 Table Filling Algorithm - step by step

A =
0 1
B =
0 1 0
A C E G C X X =
1 0 1
0 1 D X X X =
1 1 0 E X X X X =
B D F H
1 0 F X X X =
0
G X X X = X X =
1. Mark X between accepting vs. non-accepting state H X X = X X X =
2. Look 1- hop away for distinguishing states or strings A B C D E F G H

12
 Table Filling Algorithm - step by step

A =
0 1
B =
0 1 0
A C E G C X X =
1 0 1
0 1 D X X X =
1 1 0 E X X X X =
B D F H
1 0 F X X X =
0
G X X X = X X =
1. Mark X between accepting vs. non-accepting state H X X = X X X X =
2. Look 1- hop away for distinguishing states or strings A B C D E F G H

13
 Table Filling Algorithm - step by step

A =
0 1
B = =
0 1 0
A C E G C X X =
1 0 1
0 1 D X X X =
1 1 0 E X X X X =
B D F H
1 0 F X X X X X =
0
G X X X = X X =
1. Mark X between accepting vs. non-accepting state H X X = X X X X =
2. Pass 1: A B C D E F G H
Look 1- hop away for distinguishing states or strings
3. Pass 2:
Look 1-hop away again for distinguishing states or strings
continue….

14
 Table Filling Algorithm - step by step

A =
0 1
B = =
0 1 0
A C E G C X X =
1 0 1
0 1 D X X X =
1 1 0 E X X X X =
B D F H
1 0 F X X X X X =
0
G X X X = X X =
1. Mark X between accepting vs. non-accepting state H X X = X X X X =
2. Pass 1: A B C D E F G H
Look 1- hop away for distinguishing states or strings
3. Pass 2:
Look 1-hop away again for distinguishing states or strings Equivalences:
continue…. • A=B
• C=H
• D=G
15
Table Filling Algorithm - step by step

0 1 0 1

0 1 0 0 1
A C E G A C E
1 0 1 1 0
0 1 0
1 1 0 1
B D F H D F
1 0 1 0
0

Retrain only one copy for

each equivalence set of states

Equivalences:
• A=B
• C=H
• D=G
16
Table Filling Algorithm – special case

A =
0 1
B =
0 1 0
A C E G C =
1 0 1
0 1 D =
1 1 0 E ? =
B D F H
1 0 F =
0
G =
H =
A B C D E F G H

17
 How to minimize a DFA?
• Goal: Minimize the number of states in a DFA
• Algorithm:
1. Eliminate states unreachable from the start state
2. Identify and remove equivalent states
3. Output the resultant DFA

18
 Department of Computer and Science Engineering (CSE)

DFA Minimization using Equivalence Theorem

• Example
• Let us consider the following DFA −
 Department of Computer and Science Engineering (CSE)

q δ(q,0) δ(q,1)

a B C

b A D

c E F

d E F

e E f

f F f
 Department of Computer and Science Engineering (CSE)

• Let us apply the above algorithm to the above DFA −

• P0 = {(c,d,e), (a,b,f)}
• P1 = {(c,d,e), (a,b),(f)}
• P2 = {(c,d,e), (a,b),(f)}
• Hence, P1 = P2.
• There are three states in the reduced DFA. The reduced
DFA is as follows −
 Department of Computer and Science Engineering (CSE)

Q δ(q,0) δ(q,1)

(a, b) (a, b) (c,d,e)

(c,d,e) (c,d,e) (f)

(f) (f) (f)

 Department of Computer and Science Engineering (CSE)

Outcomes
• Knowledge of DFA
– Minimization of DFA
– Myhill Nerode Theorem

23
 FAQ

1. Obtain the Greibach Normal form A1àA2 A3, A2àA3 A1/ b , A3àA1A2/ a.
2. Let G = {{s , c }, {a , b }, P , s },P = S ® asb / a , A ® SbA / ss / ba .Find a derivation
tree whose yield aabbaa.

3. Find L(G),where P consists of S ® asb , s ® ab .

4. Write CFG for L(G) = {a m b n c p / m + n = p , p ³ 1}.

24
 REFERENCES
•Sipser,” Theory Of Computation, Cengage Learning.
•K.L.P MISHRA, “Introduction to computer Theory” .
•Martin J.C., “Introduction to Languages and Theory of
Computation”, Tata McGraw-Hill Publising Company Limited, 3rd
Edition.
•https://youtu.be/cFd3pz5H-TA
• https://youtu.be/qmX7DJ9j1-A
•https://en.wikipedia.org

25
 Department of Computer and Science Engineering (CSE)

UNIVERSITY INSTITUTE OF
ENGINEERING
COMPUTER SCIENCE
ENGINEERING
Bachelor of Engineering
Theory of Computation (CST-353)

Topic: Regular Language Properties

DISCOVER . LEARN .
EMPOWER
 Department of Computer and Science Engineering (CSE)

Learning Objectives & Outcomes

Objective:
• To understand the concept Regular Language and its
Properties

Outcome:
• Student will understand the
 FSM To get basic knowledge of pumping lemma
 To get knowledge of regular language
 To get knowledge of DFA
The Pumping Lemma for
Regular Sets

Proved that a given language is not

regular

3
 Pumping Lemma for Regular Sets

Let L be a regular language

Then there exists some constant N such that for
every string w  L s.t. |w|≥N, there exists a way
to break w into three parts, w=xyz, such that:
1. y≠ 
2. |xy|≤N
3. For all k≥0, all strings of the form xykz  L

This clause should hold for all regular languages.

Definition: N is called the “Pumping Lemma Constant”

4
 Pumping Lemma: Proof

• L is regular => it should have a DFA.

– Set N := number of states in the DFA
• Any string wL, s.t. |w|≥N, should have the
form: w=a1a2…am, where m≥N
• Let the states traversed after reading the first N
symbols be: {p0,p1,… pN}
 ==> There are N+1 p-states, while there are only N DFA
states
 ==> at least one state has to repeat
i.e, pi= pJwhere 0≤i<j≤N (by PHP)

5
 Pumping Lemma: Proof…

 => We should be able to break w=xyz as follows:

 x=a1a2..ai; y=ai+1ai+2..aJ; z=aJ+1aJ+2..am
 x’s path will be p0..pi
 y’s path will be pi pi+1..pJ (but pi=pJ implying a loop)
 z’s path will be pJpJ+1..pm
yk (for k loops)
 Now consider another
string wk=xykz , where k≥0 x z
p0 pi pm
 Case k=0
=pj
 DFA will reach the accept state pm
 Case k>0
 DFA will loop for yk, and finally reach the accept state pm for z
 In either case, wk L

6
 Pumping Lemma: Proof…
• For part (1): yk (for k loops)
– Since i<j, y ≠  x z
p0 pi pm
=pj
• For part (2):
– By PHP, the repetition of states has to occur within the first N
symbols in w
– ==> |xy|≤N

7
 The Purpose of the Pumping Lemma for RL

• To prove that some languages cannot be regular.

8
 How to use the pumping lemma?
Think of playing a 2 person game
– Role 1: You claim that the language cannot be
regular

– Role 2: An adversary who claims the language is

regular

– You show that the adversary’s statement will lead to a

contradiction that implyies pumping lemma cannot
hold for the language.

– You win!!

9
Note: This N can be anything (need not necessarily be the #states in the DFA.
It’s the adversary’s choice.)

Example of using the Pumping Lemma to

prove that a language is not regular
Let Leq = {w | w is a binary string with equal number of 1s
and 0s}
• Your Claim: Leq is not regular
• Proof:
 By contradiction, let Leq be regular
 P/L constant should exist
 Let N = that P/L constant
 Consider input w = 0N1N
(your choice for the template string)
 By pumping lemma, we should be able to break w=xyz, such
that:
1) y≠ 
2) |xy|≤N
3) For all k≥0, the string xykz is also in L

10
 Template string w = 0N1N = 00 …. 011 … 1
N N

Proof…
 Because |xy|≤N, xy should contain only 0s
 you
 (This and because y≠ , implies y=0+)
 Therefore x can contain at most N-1 0s
 Also, all the N 1s must be inside z
 By (3), any string of the form xykz  Leq for all k≥0
 Case k=0: xz has at most N-1 0s but has N 1s
Therefore, xy0z  Leq
Setting k=0 is
referred to as 
“pumping down”
 This violates the P/L (a contradiction)

Another way of proving this will be to show that if

Setting k>1 is the #0s is arbitrarily pumped up (e.g., k=2),
referred to as
“pumping up” then the #0s will become exceed the #1s
11
 Exercise 2

Prove L = {0n10n | n≥ 1} is not regular

Note: This n is not to be confused with the pumping

lemma constant N. That can be different.

In other words, the above question is same as proving:

– L = {0m10m | m≥ 1} is not regular

12
Closure properties of Regular
Sets

13
 Closure properties for Regular Sets (RL)
This is different

• Closure property:
from Kleene
closure
– If a set of regular languages are combined using an
operator, then the resulting language is also regular
• Regular languages are closed under:
– Union, intersection, complement, difference
– Reversal
– Kleene closure
– Concatenation
– Homomorphism Now, lets prove all of this!
– Inverse homomorphism

14
 RLs are closed under union

• IF L and M are two RLs THEN:

 they both have two corresponding regular expressions, R

and S respectively

 (L U M) can be represented using the regular expression

R+S

 Therefore, (L U M) is also regular

How can this be proved using FAs?

15
 RLs are closed under complementation

• If L is an RL over ∑, then L=∑*-L

 To show L is also regular, make the following
construction
Convert every final state into non-final, and
every non-final state into a final state

DFA for L DFA for L

qF1 qF1

q0 qi qF2 q0 qi qF2
…

…
qFk qFk

Assumes q0 is a non-final state. If not, do the opposite. 16

 RLs are closed under intersection

• A quick, indirect way to prove:

– By DeMorgan’s law:
– L ∩ M = (L U M)
– Since we know RLs are closed under union and complementation,
they are also closed under intersection
• A more direct way would be construct a finite
automaton for L ∩ M

17
 DFA construction for L ∩ M

• AL = DFA for L = {QL, ∑ , qL,FL, δL }

• AM = DFA for M = {QM, ∑ , qM,FM, δM }
• Build AL ∩ M = {QLx QM,∑, (qL,qM), FLx FM,δ} such
that:
– δ((p,q),a) = (δL(p,a), δM(q,a)), where p in QL, and q in
QM
• This construction ensures that a string w will be
accepted if and only if w reaches an accepting
state in both input DFAs.

18
 DFA construction for L ∩ M

DFA for L DFA for M

qF1 pF1

a a
q0 qi qj qF2 p0 pi pj pF2

…
DFA for LM
(qF1 ,pF1)

…
(q0 ,p0) (qi ,pi) (qj ,pj)

19
 RLs are closed under set difference

• We observe: Closed under intersection

– L-M=L∩M
Closed under
complementation
• Therefore, L - M is also regular

20
 RLs are closed under reversal
Reversal of a string w is denoted by wR
– E.g., w=00111, wR=11100
Reversal of a language:
• LR = The language generated by reversing all strings in L

Theorem: If L is regular then LR is also regular

21
  -NFA Construction for L R

New -NFA for LR

DFA for L
qF1


q0 qi
a
qj qF2  q’0 New start
state


…
Make the
old start state
as the only new qFk
final state

What to do if q0 was Reverse all transitions

one of the final states
Convert the old set of final states
in the input DFA? 22
into non-final states
 If L is regular, LR is regular (proof using
regular expressions)
• Let E be a regular expression for L
• Given E, how to build ER?
• Basis: If E= , Ø, or a, then ER=E
• Induction: Every part of E (refer to the part as “F”) can
be in only one of the three following forms:
1. F = F1+F2
• FR = F1R+F2R
2. F = F1F2
• FR = F2RF1R
3. F = (F1)*
• (FR)* = (F1R)*

23
 Department of Computer and Science Engineering (CSE)

Outcomes
• Knowledge of Regular language and pumping lemma
• Knowledge of DFA

24
 FAQ

1. Obtain the Greibach Normal form A1àA2 A3, A2àA3 A1/ b , A3àA1A2/ a.
2. Let G = {{s , c }, {a , b }, P , s },P = S ® asb / a , A ® SbA / ss / ba .Find a derivation
tree whose yield aabbaa.

3. Find L(G),where P consists of S ® asb , s ® ab .

4. Write CFG for L(G) = {a m b n c p / m + n = p , p ³ 1}.

25
 REFERENCES
•Sipser,” Theory Of Computation, Cengage Learning.
•K.L.P MISHRA, “Introduction to computer Theory” .
•Martin J.C., “Introduction to Languages and Theory of
Computation”, Tata McGraw-Hill Publising Company Limited, 3rd
Edition.
• https://youtu.be/bBSM3rs3hDw
• https://youtu.be/AG72s-aFqFs
• https://youtu.be/cFd3pz5H-TA
• https://youtu.be/qmX7DJ9j1-A
•https://en.wikipedia.org

26
 Department of Computer and Science Engineering (CSE)

UNIVERSITY INSTITUTE OF
ENGINEERING
COMPUTER SCIENCE
ENGINEERING
Bachelor of Engineering
Theory of Computation (CST-353)

Topic: Automata and Formal Languages

DISCOVER . LEARN .
EMPOWER
 Department of Computer and Science Engineering (CSE)

Learning Objectives & Outcomes

Objective:
• To understand the concept of Automata and
Formal Languages.

Outcome:
• Student will understand the
 FSM
 Moore and mealy machine
 Equivalence of Moore and Mealy Machines
Finite State Machines with Output (Mealy
and Moore Machines)
• Finite automata Machine receive input and process the
input by changing states.
• The only output that we have seen finite automata
produce so far is a yes/no at the end of processing.
• We will now look at two models of finite automata that
produce more output than a yes/no.
 Moore machine
• Basically a Moore machine is just
a FA with two extras.
1. It has TWO alphabets, an input and output alphabet.
2. It has an output letter associated with each state. The machine writes
the appropriate output letter as it enters each state.

This machine might be

considered as a
"counting" machine.

The output produced by the machine contains a 1 for each

occurrence of the substring aab found in the input string.
 Department of Computer and Science Engineering (CSE)

Moore Machine: State Transition diagram & Transition table

Transition Diagram Transition Table

 Mealy machine

• Mealy Machines are exactly as powerful as Moore machines

– (we can implement any Mealy machine using a Moore machine, and
vice versa).
• However, Mealy machines move the output function from the
state to the transition.
• This turns out to be easier to deal with in practice, making
Mealy machines more practical.
 Department of Computer and Science Engineering (CSE)

Mealy Machine: State Transition diagram & Transition table

Transition Diagram Transition Table

 A Mealy machine produces output on a
transition instead of on entry into a state
• Transitions are labelled i/o where
– i is a character in the input alphabet and
– o is a character in the output alphabet.

The following Mealy machine takes the one's

complement of its binary input. In other words, it
flips each digit from a 0 to a 1 or from a 1 to a 0.

• Mealy machines are complete in the sense that there is a

transition for each character in the input alphabet leaving
every state.
• There are no accept states in a Mealy machine
 Automata networks

• (Parts of the) output of a Mealy/Moore automaton can be

input to another automaton
– Pipeline
– Dataflow architecture
– Feedback network
 Automata as transition systems

• Output string can be seen as a

characterization of what holds
in a particular state Example transition system
• Output language e.g.
propositional logic
• Given a set of atomic p,q 1
propositions, states are labelled
with propositions that “hold” in
a state
(Moore style) q 2 3 p
 Example: Parity checker

• Serial input string

– OUT=1 if odd # of 1s in input
– OUT=0 if even # of 1s in input
• Let’s do this for Moore and Mealy

11
 Department of Computer and Science Engineering (CSE)

Equivalence of Moore and Mealy Machines

Conversion from Mealy machine to Moore Machine
• In Moore machine, the output is associated with every state, and in
Mealy machine, the output is given along the edge with input symbol.
To convert Moore machine to Mealy machine, state output symbols
are distributed to input symbol paths. But while converting the
Mealy machine to Moore machine, we will create a separate state for
every new output symbol and according to incoming and outgoing
edges are distributed.
The following steps are used for converting Mealy machine to the Moore
machine:
 Step 1: For each state(Qi), calculate the number of different outputs that are
available in the transition table of the Mealy machine.
 Step 2: Copy state Qi, if all the outputs of Qi are the same. Break qi into n states
as Qin, if it has n distinct outputs where n = 0, 1, 2....
 Step 3: If the output of initial state is 0, insert a new initial state at the starting
which gives 1 output.
 Department of Computer and Science Engineering (CSE)

Conversion from Mealy machine to Moore Machine

• Example 1:
• Convert the following Mealy machine into equivalent
Moore machine.
 Department of Computer and Science Engineering (CSE)

• Solution:
• Transition table for above Mealy machine is as follows:
 Department of Computer and Science Engineering (CSE)

• For state q1, there is only one incident edge with output
0. So, we don't need to split this state in Moore machine.
• For state q2, there is 2 incident edge with output 0 and 1.
So, we will split this state into two states q20( state with
output 0) and q21(with output 1).
• For state q3, there is 2 incident edge with output 0 and 1.
So, we will split this state into two states q30( state with
output 0) and q31( state with output 1).
• For state q4, there is only one incident edge with output
0. So, we don't need to split this state in Moore machine.
• Transition table for Moore machine will be:
Department of Computer and Science Engineering (CSE)
 Department of Computer and Science Engineering (CSE)

Transition diagram for Moore machine will be:

 Department of Computer and Science Engineering (CSE)

Conversion from Moore machine to Mealy Machine

• Example 1:
• Convert the following Moore machine into its equivalent
Mealy machine.
 Department of Computer and Science Engineering (CSE)

Solution:
The transition table of given Moore machine is as follows:

Q A b Output(λ)

q0 q0 q1 0

q1 q0 q1 1
 Department of Computer and Science Engineering (CSE)

• The equivalent Mealy machine can be obtained as

follows:
• λ' (q0, a) = λ(δ(q0, a))
• = λ(q0)
• =0

• λ' (q0, b) = λ(δ(q0, b))

• = λ(q1)
• =1
 Department of Computer and Science Engineering (CSE)

• The λ for state q1 is as follows:

• λ' (q1, a) = λ(δ(q1, a))
• = λ(q0)
• =0

• λ' (q1, b) = λ(δ(q1, b))

• = λ(q1)
• =1
 Department of Computer and Science Engineering (CSE)

Hence the transition table for the Mealy machine can be

drawn as follows:
 Department of Computer and Science Engineering (CSE)

The equivalent Mealy machine will be,

 Department of Computer and Science Engineering (CSE)

Outcomes

• Knowledge of drawing state diagram of Mealy Machine.

• Knowledge of drawing state diagram of Moore Machine.
• Equivalence of Moore and Mealy Machine.
 Department of Computer and Science Engineering (CSE)

FAQ :

1. Construct a finite automata that accepts {0,1}+.

2. Eliminate the e production from the CFG given below
• A-> 0 B 1 | 1 B 1
• B-> 0 B | 1 B | e
3. Find the derivation tree for the grammar, G = {{s , A , B },
{a , b }, P , s } ,whereP = {S / bA / bB , A / ab , B / aBb /
a}
 Department of Computer and Science Engineering (CSE)

REFERENCES :

• Martin J.C., “Introduction to Languages and Theory of

Computation”, Tata McGraw-Hill Publising Company
Limited, 3rd Edition.
• Hopcroft J.E. and Ullman J.D., “Introduction to Automata
Theory Languages and Computation”, Narosa
Publications.
• https://www.javatpoint.com/conversion-from-mealy-
machine-to-moore-machine
• https://www.javatpoint.com/conversion-from-moore-
machine-to-mealy-machine
 Department of Computer and Science Engineering (CSE)

UNIVERSITY INSTITUTE OF
ENGINEERING
COMPUTER SCIENCE
ENGINEERING
Bachelor of Engineering
Theory of Computation (CST-353)

Topic: Context Free Language

DISCOVER . LEARN .
EMPOWER
 Department of Computer and Science Engineering (CSE)

Learning Objectives & Outcomes

Objective:
• To understand the concept of CFL, Parse tree.

Outcome:
• Student will understand the
 To know about CFG and its definition
 Department of Computer and Science Engineering (CSE)

Context Free Grammar

• A context-free grammar (CFG) is a set of recursive
rewriting rules (or productions) used to generate
patterns of strings.
Context-free grammar G can be defined by four tuples as:
• G = (V, T, P, S)
Where,
• G is the grammar.
• V is the final set of a non-terminal symbol. Like A,B,C etc
• T is the final set of a terminal symbol. like; a, b, c,1 , 2 etc.
• P is a set of production rules.
• S is the start symbol which is used to derive the string.
 Department of Computer and Science Engineering (CSE)

Examples of Context Free Grammar:

• {0n1n:n ≥ 0}{0n1n: n ≥ 0} is context-free because it is

generated by the context-free grammar ({S}, {0,1}, R,S)
({S}, {0,1}, P, S), where the set of rules, P, is as;

• S→0S1∣ε
(ε to denote the empty or null string.)
 Department of Computer and Science Engineering (CSE)

Example #2

• Language of balanced paranthesis

e.g., ()(((())))((()))….
• CFG?

G:
S => (S) | SS | 

How would you “interpret” the string “(((()))()())” using this grammar?

5
 Department of Computer and Science Engineering (CSE)

Example #3

• A grammar for L = {0m1n | m≥n}

• CFG?
G:
S => 0S1 | A
A => 0A | 

How would you interpret the string “00000111”

using this grammar?

6
 Department of Computer and Science Engineering (CSE)

Applications of CFLs & CFGs

• Compilers use parsers for syntactic checking
• Parsers can be expressed as CFGs
1. Balancing paranthesis:
• B ==> BB | (B) | Statement
• Statement ==> …
2. If-then-else:
• S ==> SS | if Condition then Statement else Statement | if Condition then
Statement | Statement
• Condition ==> …
• Statement ==> …
3. C paranthesis matching { … }
4. Pascal begin-end matching
5. YACC (Yet Another Compiler-Compiler)

7
 Department of Computer and Science Engineering (CSE)

Derivation trees

Derivation tree is a graphical representation for the

derivation of the given production rules for a given CFG. It is
the simple way to show how the derivation can be done to
obtain some string from a given set of production rules. The
derivation tree is also called a parse tree.

A parse tree contains the following properties:

• The root node is always a node indicating start symbols.
• The derivation is read from left to right.
• The leaf node is always terminal nodes.
• The interior nodes are always the non-terminal nodes.
 Department of Computer and Science Engineering (CSE)

Example :
Production rules:
• E=E+E
• E=E*E
• E=a|b|c

Input
• a*b+c
 Department of Computer and Science Engineering (CSE)

Step 1:

Step 2:
 Department of Computer and Science Engineering (CSE)

Step 3:

Step 4:
 Department of Computer and Science Engineering (CSE)

Step 5:
 Department of Computer and Science Engineering (CSE)

Left-most & Right-most Derivation Styles

G:
E => E+E | E*E | (E) | F
F => aF | bF | 0F | 1F | 
Derive the string a*(ab+10) from G: E =*=>G a*(ab+10)
E E
==> E * E ==> E * E

==> F * E ==> E * (E)

Left-most ==> aF * E ==> E * (E + E) Right-most
derivation: ==> a * E ==> E * (E + F)
derivation:
==> a * (E) ==> E * (E + 1F)

Always ==> a * (E + E) ==> E * (E + 10F) Always

substitute ==> a * (F + E) ==> E * (E + 10)
substitute
leftmost ==> a * (aF + E) ==> E * (F + 10)
rightmost
==> a * (abF + E) ==> E * (aF + 10)
variable variable
==> a * (ab + E) ==> E * (abF + 0)

==> a * (ab + F) ==> E * (ab + 10)

==> a * (ab + 1F) ==> F * (ab + 10)

==> a * (ab + 10F) ==> aF * (ab + 10)

==> a * (ab + 10) ==> a * (ab + 10)

13
 Department of Computer and Science Engineering (CSE)

Leftmost vs. Rightmost derivations

Q1) For every leftmost derivation, there is a rightmost

derivation, and vice versa. True or False?

True - will use parse trees to prove this

Q2) Does every word generated by a CFG have a leftmost

and a rightmost derivation?

Yes – easy to prove (reverse direction)

Q3) Could there be words which have more than one

leftmost (or rightmost) derivation?

Yes – depending on the grammar

14
Parse trees

15
 Department of Computer and Science Engineering (CSE)

Parse Trees
• Each CFG can be represented using a parse tree:
– Each internal node is labeled by a variable in V
– Each leaf is terminal symbol
– For a production, A==>X1X2…Xk, then any internal node labeled A
has k children which are labeled from X1,X2,…Xk from left to right

Parse tree for production and all other subsequent productions:

A ==> X1..Xi..Xk A

X1 … Xi … Xk

16
 Examples

Recursive inference
A
E + E
0 A 0
F F

Derivation
1 A 1
a 1


Parse tree for 0110

Parse tree for a + 1
G: G:
E => E+E | E*E | (E) | F A => 0A0 | 1A1 | 0 | 1 | 
F => aF | bF | 0F | 1F | 0 | 1 | a | b
17
Parse Trees, Derivations, and Recursive
Inferences
Production:
A ==> X1..Xi..Xk
A

Derivation
… …
Recursive

X1 Xi Xk
inference

Left-most Parse tree

derivation

Derivation Right-most
Recursive
derivation
inference
18
 Department of Computer and Science Engineering (CSE)

Interchangeability of different CFG representations

• Parse tree ==> left-most derivation

– DFS left to right
• Parse tree ==> right-most derivation
– DFS right to left
• ==> left-most derivation == right-most
derivation
• Derivation ==> Recursive inference
– Reverse the order of productions
• Recursive inference ==> Parse trees
– bottom-up traversal of parse tree

19
 Department of Computer and Science Engineering (CSE)

Outcomes

• Knowledge of applications of CFLs & CFGs

• Knowledge of CFG
• Knowledge of parse trees
 Department of Computer and Science Engineering (CSE)

FAQ :

1. Find L(G),where P consists of S ® asb , s ® ab .

2. What are the closure properties of context- free
Language?
3. Write CFG for L(G) = {a m b n c p / m + n = p , p ³ 1}.
 Department of Computer and Science Engineering (CSE)

REFERENCES :

• Martin J.C., “Introduction to Languages and Theory of

Computation”, Tata McGraw-Hill Publising Company
Limited, 3rd Edition.
• Daniel I.A. Cohen, “Introduction to computer Theory”, John
Wiley.
• K.L.P MISHRA, “Introduction to computer Theory” .
• https://youtu.be/NNSK4570Kho
• https://youtu.be/A_Zj-WaA2Lo
THANK YOU

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

UNIVERSITY INSTITUTE OF
ENGINEERING
COMPUTER SCIENCE
ENGINEERING
Bachelor of Engineering
Theory of Computation (CST-353)

Topic: CNF and GNF

DISCOVER . LEARN .
EMPOWER
 Department of Computer and Science Engineering (CSE)

Learning Objectives & Outcomes

Objective:
• To understand the concept of CNF and GNF.

Outcome:
• Student will understand the
 CNF
 GNF
 Chomsky Normal Form
• A context free grammar is said to be in Chomsky Normal
Form if all productions are in the following form:
A → BC
A→α
• A, B and C are non terminal symbols
• α is a terminal symbol
 Department of Computer and Science Engineering (CSE)

Preliminary Simplifications

There are three preliminary simplifications

1 Eliminate Useless Symbols

2 Eliminate ε productions

3 Eliminate unit productions

 Department of Computer and Science Engineering (CSE)

Eliminate Useless Symbols:

We need to determine if the symbol is useful by

identifying if a symbol is generating and is reachable

• X is generating if X 
* ω for some terminal string ω.
• X is reachable if there is a derivation X 
* αXβ
for some α and β
 Department of Computer and Science Engineering (CSE)

Example: Removing non-generating symbols

S → AB | a
A→b Initial CFL grammar

S → AB | a
Identify generating symbols
A→b

S→a
Remove non-generating
A→b
 Department of Computer and Science Engineering (CSE)

Example: Removing non-reachable symbols

S→a
Identify reachable symbols
A→b

S→a Eliminate non-reachable

 Department of Computer and Science Engineering (CSE)

The order is important.

Looking first for non-reachable symbols and then

for non-generating symbols can still leave some
useless symbols.

S → AB | a S→a
A→b A→b
 Department of Computer and Science Engineering (CSE)

Finding generating symbols

If there is a production A → α, and every symbol

of α is already known to be generating. Then A is
generating

We cannot use S → AB because

S → AB | a B has not been established to
A→b be generating
 Department of Computer and Science Engineering (CSE)

Finding reachable symbols

S is surely reachable. All symbols in the body of a

production with S in the head are reachable.

S → AB | a In this example the symbols

A→b {S, A, B, a, b} are reachable.
 Department of Computer and Science Engineering (CSE)

Eliminate ε Productions

• In a grammar ε productions are convenient but

not essential
• If L has a CFG, then L – {ε} has a CFG

A 
* ε

Nullable variable
 Department of Computer and Science Engineering (CSE)

If A is a nullable variable

• Whenever A appears on the body of a production

A might or might not derive ε

S → ASA | aB
A→B|S Nullable: {A, B}
B→b|ε
 Department of Computer and Science Engineering (CSE)

Eliminate ε Productions

• Create two version of the production, one with

the nullable variable and one without it
• Eliminate productions with ε bodies

S → ASA | aB S → ASA | aB | AS | SA | S | a
A→B|S A→B|S
B→b|ε B→b
 Department of Computer and Science Engineering (CSE)

Eliminate ε Productions

• Create two version of the production, one with

the nullable variable and one without it
• Eliminate productions with ε bodies

S → ASA | aB S → ASA | aB | AS | SA | S | a
A→B|S A→B|S
B→b|ε B→b
 Department of Computer and Science Engineering (CSE)

Eliminate ε Productions

• Create two version of the production, one with

the nullable variable and one without it
• Eliminate productions with ε bodies

S → ASA | aB S → ASA | aB | AS | SA | S | a
A→B|S A→B|S
B→b|ε B→b
 Department of Computer and Science Engineering (CSE)

Eliminate unit productions

A unit production is one of the form A → B where both

A and B are variables

Identify unit pairs

A 
* B

A → B, B → ω, then A → ω
 Department of Computer and Science Engineering (CSE)

Example:
T = {*, +, (, ), a, b, 0, 1} Pairs Productions
( E, E ) E→E+T
I → a | b | Ia | Ib | I0 | I1 ( E, T ) E→T*F
F → I | (E) ( E, F ) E → (E)
T→F|T*F ( E, I ) E → a | b | Ia | Ib | I0 | I1
E→T|E+T ( T, T ) T→T*F
( T, F ) T → (E)
( T, I ) T → a | b | Ia |Ib | I0 | I1
Basis: (A, A) is a unit pair ( F, F ) F → (E)
of any variable A, if ( F, I ) F → a | b | Ia | Ib | I0 | I1
A  * A by 0 steps. ( I, I ) I → a | b | Ia | Ib | I0 | I1
 Department of Computer and Science Engineering (CSE)

Example:
Pairs Productions
… …
( T, T ) T→T*F
( T, F ) T → (E)
( T, I ) T → a | b | Ia |Ib | I0 | I1
… …

I → a | b | Ia | Ib | I0 | I1
E → E + T | T * F | (E ) | a | b | la | lb | l0 | l1
T → T * F | (E) | a | b | Ia | Ib | I0 | I1
F → (E) | a | b | Ia | Ib | I0 | I1
 Department of Computer and Science Engineering (CSE)

Final Simplification
Chomsky Normal Form (CNF)

Starting with a CFL grammar with the preliminary

simplifications performed

1. Arrange that all bodies of length 2 or more to

consists only of variables.
2. Break bodies of length 3 or more into a cascade of
productions, each with a body consisting of two
variables.
 Department of Computer and Science Engineering (CSE)

Step 1: For every terminal α that appears in a body

of length 2 or more create a new variable that has
only one production.
E → E + T | T * F | (E ) | a | b | la | lb | l0 | l1
T → T * F | (E) | a | b | Ia | Ib | I0 | I1
F → (E) | a | b | Ia | Ib | I0 | I1
I → a | b | Ia | Ib | I0 | I1
E → EPT | TMF | LER | a | b | lA | lB | lZ | lO
T → TMF | LER | a | b | IA | IB | IZ | IO
F → LER | a | b | IA | IB | IZ | IO
I → a | b | IA | IB | IZ | IO
A→a B→b Z→0 O→1
P→+ M→* L→( R→)
 Department of Computer and Science Engineering (CSE)

Step 2: Break bodies of length 3 or more adding

more variables

E → EPT | TMF | LER | a | b | lA | lB | lZ | lO

T → TMF | LER | a | b | IA | IB | IZ | IO
C1 → PT
F → LER | a | b | IA | IB | IZ | IO
C2 → MF
I → a | b | IA | IB | IZ | IO
C3 → ER
A→a B→b Z→0 O→1
P→+ M→* L→( R→)
 Department of Computer and Science Engineering (CSE)

Greibach Normal Form

A context free grammar is said to be in Greibach

Normal Form if all productions are in the following
form:

A → αX

• A is a non terminal symbols

• α is a terminal symbol
• X is a sequence of non terminal symbols.
It may be empty.
 Department of Computer and Science Engineering (CSE)

Example:

S → XA | BB S = A1 A1 → A2A3 | A4A4
B → b | SB X = A2 A4 → b | A1A4
X→b A = A3 A2 → b
A→a B = A4 A3 → a

CNF New Labels Updated CNF

 Department of Computer and Science Engineering (CSE)

Example:

A1 → A2A3 | A4A4 First Step Ai → AjXk j > i

A4 → b | A1A4
A2 → b Xk is a string of zero
A3 → a or more variables

A4 → A1A4
 Department of Computer and Science Engineering (CSE)

Example:

First Step Ai → AjXk j > i

A4 → A1A4 A1 → A2A3 | A4A4

A4 → A2A3A4 | A4A4A4 | b A4 → b | A1A4
A2 → b
A4 → bA3A4 | A4A4A4 | b
A3 → a
 Department of Computer and Science Engineering (CSE)

Example:

A1 → A2A3 | A4A4 Second Step

A4 → bA3A4 | A4A4A4 | b
Eliminate Left
A2 → b
Recursions
A3 → a

A4 → A4A4A4
 Department of Computer and Science Engineering (CSE)

Example:
Second Step
Eliminate Left
Recursions

A4 → bA3A4 | b | bA3A4Z | bZ A1 → A2A3 | A4A4

Z → A4A4 | A4A4Z A4 → bA3A4 | A4A4A4 | b
A2 → b
A3 → a
 Department of Computer and Science Engineering (CSE)

Example:

A1 → A2A3 | A4A4
A4 → bA3A4 | b | bA3A4Z | bZ A → αX
Z → A4A4 | A4A4 Z
A2 → b GNF
A3 → a
 Department of Computer and Science Engineering (CSE)

Example:
A1 → A2A3 | A4A4
A4 → bA3A4 | b | bA3A4Z | bZ
Z → A4A4 | A4A4 Z
A2 → b
A3 → a

A1 → bA3 | bA3A4A4 | bA4 | bA3A4ZA4 | bZA4

Z → bA3A4A4 | bA4 | bA3A4ZA4 | bZA4 | bA3A4A4 | bA4 | bA3A4ZA4 | bZA4
 Department of Computer and Science Engineering (CSE)

Example:

A1 → bA3 | bA3A4A4 | bA4 | bA3A4ZA4 | bZA4

A4 → bA3A4 | b | bA3A4Z | bZ
Z → bA3A4A4 | bA4 | bA3A4ZA4 | bZA4 | bA3A4A4 | bA4 | bA3A4ZA4 | bZA4
A2 → b
A3 → a

Grammar in Greibach Normal Form

 Department of Computer and Science Engineering (CSE)

Summary (Some properties)

• Every CFG that doesn’t generate the empty string
can be simplified to the Chomsky Normal Form and
Greibach Normal Form
• The derivation tree in a grammar in CNF is a binary
tree
• In the GNF, a string of length n has a derivation of
exactly n steps
• Grammars in normal form can facilitate proofs
• CNF is used as starting point in the algorithm CYK
 Department of Computer and Science Engineering (CSE)

FAQ :
1. Convert the following grammar to the Chomsky Normal Form.
S→P
P → aPb | ε
2. Is the following grammar context-free?
S → aBSc | abc
Ba → aB
Bb → bb
3. Prove that the language L = { anbncn | n ≥ 1 } is not context-free.
4. Convert the following grammar to the Greibach Normal Form.
S -> a | CD | CS
A -> a | b | SS
C -> a
D -> AS
 Department of Computer and Science Engineering (CSE)

REFERENCES :

• Martin J.C., “Introduction to Languages and Theory of

Computation”, Tata McGraw-Hill Publising Company
Limited, 3rd Edition.
• Hopcroft J.E. and Ullman J.D., “Introduction to Automata
Theory Languages and Computation”, Narosa
Publications.
• https://www.erode-sengunthar.ac.in/wp-
content/uploads/2019/04/NORMAL-FORMS.ppt
 Department of Computer and Science Engineering (CSE)

UNIVERSITY INSTITUTE OF
ENGINEERING
COMPUTER SCIENCE
ENGINEERING
Bachelor of Engineering
Theory of Computation (CST-353)

Topic: Context Free Language

DISCOVER . LEARN .
EMPOWER
 Department of Computer and Science Engineering (CSE)

Learning Objectives & Outcomes

Objective:
• To understand the concept of context free languages.

Outcome:
• Student will understand the
 To know about Simplifying CFGs, Normal forms
 Department of Computer and Science Engineering (CSE)

Regular Grammar

• Type 0 known as Unrestricted Grammar.

• Type 1 known as Context Sensitive Grammar.
• Type 2 known as Context Free Grammar.
• Type 3 Regular Grammar.
Department of Computer and Science Engineering (CSE)
How to “simplify” CFGs?

5
 Three ways to simplify/clean a CFG

1. Eliminate useless symbols

2. Eliminate -productions

3. Eliminate unit productions

6
 Eliminating useless symbols
A symbol X is reachable if there exists:
– S *  X 

A symbol X is generating if there exists:

– X * w,
• for some w  T*

For a symbol X to be “useful”, it has to be both reachable and

generating
• S *  X  * w’, for some w’  T*

7
 Algorithm to detect useless
1.
symbols
First, eliminate all symbols that are not generating

2. Next, eliminate all symbols that are not reachable

8
 Example: Useless symbols
• SAB | a
• A b

1. A, S are generating
2. B is not generating (and therefore B is useless)
3. ==> Eliminating B… (i.e., remove all productions that involve B)
1. S a
2. A  b
4. Now, A is not reachable and therefore is useless

5. Simplified G:
1. S  a What would happen if you reverse the order:
i.e., test reachability before generating?
Will fail to remove: A
b
9
 X * w
Algorithm to find all generating symbols
• Given: G=(V,T,P,S)
• Basis:
– Every symbol in T is obviously generating.
• Induction:
– Suppose for a production A , where  is generating
– Then, A is also generating

10
 S *  X 
Algorithm to find all reachable symbols
• Given: G=(V,T,P,S)
• Basis:
– S is obviously reachable (from itself)
• Induction:
– Suppose for a production A 1 2… k, where A is reachable
– Then, all symbols on the right hand side, {1, 2 ,… k} are also
reachable.

11
Eliminating -productions

A => 

12
 Eliminating -productions
It is not possible to eliminate -productions for languages
which include  in their word set

Theorem: If G=(V,T,P,S) is a CFG for a language L, then L\ {}

has a CFG without -productions

Definition: A is “nullable” if A* 

• If A is nullable, then any production of the form
“B CAD” can be simulated by:
• B  CD | CAD
– This can allow us to remove  transitions for A

13
 Algorithm to detect all nullable
• Basis:
variables
– If A  is a production in G, then A is nullable
(note: A can still have other productions)
• Induction:
– If there is a production B C1C2…Ck, where every Ci is nullable, then
B is also nullable

14
 Eliminating -productions
Given: G=(V,T,P,S)
Algorithm:
1. Detect all nullable variables in G
2. Then construct G1=(V,T,P1,S) as follows:
i. For each production of the form: AX1X2…Xk, where k≥1,
suppose m out of the k Xi’s are nullable symbols
ii. Then G1 will have 2m versions for this production
i. i.e, all combinations where each Xi is either present or absent
iii. Alternatively, if a production is of the form: A, then
remove it

15
 Example: Eliminating -productions
• Let L be the language represented by the following CFG G:
i. SAB
ii. AaAA | 
iii. BbBB | 
Simplified
Goal: To construct G1, which is the grammar for L-{} grammar

• Nullable symbols: {A, B}

• G1 can be constructed from G as follows:

– B  b | bB | bB | bBB G1:
• ==> B  b | bB | bBB • S  A | B | AB
– Similarly, A  a | aA | aAA • A  a | aA | aAA
– Similarly, S  A | B | AB • B  b | bB | bBB
+
• Note: L(G) = L(G1) U {}
• S

16
 Eliminating unit productions

A => B B has to be a variable

What’s the point of removing unit transitions ? Will save #substitutions

E.g., A=>B | … A=>xxx | yyy | zzz | …

B=>C | … B=> xxx | yyy | zzz | …
C=>D | … C=> xxx | yyy | zzz | …
D=>xxx | yyy | zzz D=>xxx | yyy | zzz
17
before after
 AB
Eliminating unit productions
• Unit production is one which is of the form A B, where both A & B are
variables
• E.g.,
1. E  T | E+T
2. T  F | T*F
3. F  I | (E)
4. I  a | b | Ia | Ib | I0 | I1
– How to eliminate unit productions?

• Replace E T with E  F | T*F

• Then, upon recursive application wherever there is a unit production:

– E F | T*F | E+T (substituting for T)
– E I | (E) | T*F| E+T (substituting for F)
– E a | b | Ia | Ib | I0 | I1 | (E) | T*F | E+T (substituting for I)
– Now, E has no unit productions

• Similarly, eliminate for the remainder of the unit productions

18
 The Unit Pair Algorithm: to remove unit productions

• Suppose AB1 B2  …  Bn  

• Action: Replace all intermediate productions to produce  directly
– i.e., A ; B1 ; … Bn  ;

Definition: (A,B) to be a “unit pair” if A*B

• We can find all unit pairs inductively:

– Basis: Every pair (A,A) is a unit pair (by definition). Similarly, if AB is a
production, then (A,B) is a unit pair.

– Induction: If (A,B) and (B,C) are unit pairs, and AC is also a unit pair.

19
 The Unit Pair Algorithm: to remove unit productions

Input: G=(V,T,P,S)
Goal: to build G1=(V,T,P1,S) devoid of unit
productions
Algorithm:
1. Find all unit pairs in G
2. For each unit pair (A,B) in G:
1. Add to P1 a new production A, for every B which is a
non-unit production
2. If a resulting production is already there in P, then there is no
need to add it.

20
 Example: eliminating unit
productions
Unit pairs Only non-unit
productions to be
added to P1
G:
1. E  T | E+T (E,E) E  E+T
2. T  F | T*F
3. F  I | (E) (E,T) E  T*F
4. I  a | b | Ia | Ib | I0 | I1 (E,F) E  (E)
(E,I) E  a|b|Ia | Ib | I0 | I1
(T,T) T  T*F
(T,F) T  (E)
(T,I) T  a|b| Ia | Ib | I0 | I1
G1:
1. E  E+T | T*F | (E) | a| b | Ia | Ib | I0 | I1 (F,F) F  (E)
2. T  T*F | (E) | a| b | Ia | Ib | I0 | I1
3. F  (E) | a| b | Ia | Ib | I0 | I1 (F,I) F  a| b| Ia | Ib | I0 |
4. I  a | b | Ia | Ib | I0 | I1 I1
(I,I) I  a| b | Ia | Ib | I0 |
I1
21
 Putting all this together…
• Theorem: If G is a CFG for a language that contains at
least one string other than , then there is another CFG
G1, such that L(G1)=L(G) - , and G1 has:
– no  -productions
– no unit productions
– no useless symbols

• Algorithm:
Step 1) eliminate  -productions
Step 2) eliminate unit productions
Step 3) eliminate useless symbols

22
 Why normal forms?
• If all productions of the grammar could be
expressed in the same form(s), then:

a. It becomes easy to design algorithms that use the

grammar

b. It becomes easy to show proofs and properties

23
 Chomsky Normal Form (CNF)
Let G be a CFG for some L-{}
Definition:
G is said to be in Chomsky Normal Form if all its
productions are in one of the following two
forms:
i. A  BC where A,B,C are variables, or
ii. Aa where a is a terminal
– G has no useless symbols
– G has no unit productions
– G has no -productions
24
 CNF checklist
Is this grammar in CNF?
G1:
1. E  E+T | T*F | (E) | Ia | Ib | I0 | I1
2. T  T*F | (E) | Ia | Ib | I0 | I1
3. F  (E) | Ia | Ib | I0 | I1
4. I  a | b | Ia | Ib | I0 | I1

Checklist:
• G has no -productions
• G has no unit productions
• G has no useless symbols
• But…
• the normal form for productions is violated

So, the grammar is not in CNF

25
 How to convert a G into CNF?
• Assumption: G has no -productions, unit productions or useless symbols

1) For every terminal a that appears in the body of a production:

i. create a unique variable, say Xa, with a production Xa  a, and
ii. replace all other instances of a in G by Xa

2) Now, all productions will be in one of the following two

forms:
– A  B1B2… Bk (k≥3) or Aa

3) Replace each production of the form A  B1B2B3… Bk by:

B2 C2
– AB1C1 C1B2C2 … Ck-3Bk-2Ck-2 Ck-2Bk-1Bk
B1 C1

26
 Example #1

G in CNF:
G:
X0 => 0
S => AS | BABC
X1 => 1
A => A1 | 0A1 | 01
S => AS | BY1
B => 0B | 0 Y1 => AY2
C => 1C | 1 Y2 => BC
A => AX1 | X0Y3 | X0X1
Y3 => AX1
B => X0B | 0
C => X1C | 1

All productions are of the form: A=>BC or A=>a

27
 Example #2

G: 1. E  EX+T | TX*F | X(EX) | IXa | IXb | IX0 | IX1

1. E  E+T | T*F | (E) | Ia | Ib | I0 | I1 2. T  TX*F | X(EX) | IXa | IXb | IX0 | IX1
2. T  T*F | (E) | Ia | Ib | I0 | I1 3. F  X(EX) | IXa | IXb | IX0 | IX1
3. F  (E) | Ia | Ib | I0 | I1 4. I  Xa | Xb | IXa | IXb | IX0 | IX1
4. I  a | b | Ia | Ib | I0 | I1 Step (1) 5. X+  +
6. X*  *
7. X+  +
8. X(  (
9. …….

1. E  EC1 | TC2 | X(C3 | IXa | IXb | IX0 | IX1

2. C1  X+T
3. C2  X*F
4. C3  EX)
5. T  ..…….
6. ….

28
 Languages with 

• For languages that include ,

– Write down the rest of grammar in CNF
– Then add production “S => ” at the end

G in CNF:
E.g., consider:
X0 => 0
G:
X1 => 1
S => AS | BABC
S => AS | BY1 | 
A => A1 | 0A1 | 01 | 
B => 0B | 0 |  Y1 => AY2
C => 1C | 1 |  Y2 => BC

A => AX1 | X0Y3 | X0X1

Y3 => AX1
B => X0B | 0

29
C => X1C | 1
 Other Normal Forms

• Griebach Normal Form (GNF)

– All productions of the form
A==>a 

30
Return of the Pumping Lemma !!

Think of languages that cannot be CFL

== think of languages for which a stack will not be enough

e.g., the language of strings of the form ww

31
 Why pumping lemma?

• A result that will be useful in proving languages that are

not CFLs
– (just like we did for regular languages)

• But before we prove the pumping lemma for CFLs ….

– Let us first prove an important property about parse trees

32
 Observe that any parse tree generated by a CNF will be a
binary tree, where all internal nodes have exactly two children
(except those nodes connected to the leaves).

The “parse tree theorem”

Given: Parse tree for w
– Suppose we have a S = A0
parse tree for a string
w, according to a CNF A1
grammar, G=(V,T,P,S) A2
h
– Let h be the height of = tree height

the parse tree

Implies: Ah-1
– |w| ≤ 2h-1
In other words, a CNF parse tree’s string yield (w) a
can no longer be 2h-1
w
33
 To show: |w| ≤ 2h-1

Proof…The size of parse trees

Proof: (using induction on h) Parse tree for w
Basis: h = 1
 Derivation will have to be “Sa” S = A0
 |w|= 1 = 21-1 .

Ind. Hyp: h = k-1 A B

 |w|≤ 2k-2

h
Ind. Step: h = k = height
S will have exactly two children:
SAB

 Heights of A & B subtrees are at

most h-1

 w = wA wB, where |wA| ≤ 2k-2 and

|wB| ≤ 2k-2
wA wB
 |w| ≤ 2k-1
w
34
Implication of the Parse Tree Theorem (assuming CNF)

Fact:
– If the height of a parse tree is h, then
• ==> |w| ≤ 2h-1

Implication:
– If |w| ≥ 2h, then
• Its parse tree’s height is at least h+1

35
 The Pumping Lemma for CFLs
Let L be a CFL.
Then there exists a constant N, s.t.,
– if z L s.t. |z|≥N, then we can write z=uvwxy, such that:
1. |vwx| ≤ N
2. vx≠
3. For all k≥0: uvkwxky  L

36
 Proof: Pumping Lemma for CFL
• If L=Φ or contains only , then the lemma is
trivially satisfied (as it cannot be violated)

• For any other L which is a CFL:

– Let G be a CNF grammar for L
– Let m = number of variables in G
– Choose N=2m.
– Pick any z  L s.t. |z|≥ N
 the parse tree for z should have a height ≥ m+1
(by the parse tree theorem)

37
 Meaning:
Repetition in the
Parse tree for z last m+1 variables

h-m≤ i < j ≤ h
S = A0 S = A0
+

A1 Ai = Aj
A2 Ai
h ≥ m+1 h ≥ m+1

Aj
m+1
Ah-1
u v x y
Ah=a
w

z z = uvwxy
• Therefore, vx≠
38
 Extending the parse tree…

S = A0
S = A0

Replacing Or, replacing

Aj with Ai Ai with Aj
(k times) Aj
Ai=Aj
h ≥ m+1

Ai
w

Ai u y
u v x y
z = uwy
v x
w ==> For all k≥0: uvkwxky L

z = uvkwxky 39
 Proof contd..

• Also, since Ai’s subtree no taller than m+1

==> the string generated under Ai‘s subtree, which is vwx,

cannot be longer than 2m (=N)

But, 2m =N

==> |vwx| ≤ N

This completes the proof for the pumping lemma.

40
 Application of Pumping Lemma for CFLs

Example 1: L = {ambmcm | m>0 }

Claim: L is not a CFL
Proof:
– Let N <== P/L constant
– Pick z = aNbNcN
– Apply pumping lemma to z and show that there exists at
least one other string constructed from z (obtained by
pumping up or down) that is  L

41
 Proof contd…
– z = uvwxy
– As z = aNbNcN and |vwx| ≤ N and vx≠
• ==> v, x cannot contain all three symbols (a,b,c)
• ==> we can pump up or pump down to build another string
which is  L

42
 Example #2 for P/L application
• L = { ww | w is in {0,1}*}

• Show that L is not a CFL

– Try string z = 0N0N

• what happens?
– Try string z = 0N1N0N1N
• what happens?

43
 Example 3
2
• L={ 0k | k is any integer)

• Prove L is not a CFL using Pumping Lemma

44
 Example 4

• L = {aibjck | i<j<k }

• Prove that L is not a CFL

45
CFL Closure Properties

46
 Closure Property Results
• CFLs are closed under:
– Union
– Concatenation
– Kleene closure operator
– Substitution
– Homomorphism, inverse homomorphism
– reversal
• CFLs are not closed under: Note: Reg languages
– Intersection are closed
under
– Difference these
– Complementation operators

47
 Strategy for Closure Property Proofs
• First prove “closure under substitution”
• Using the above result, prove other closure properties
• CFLs are closed under:
– Union
– Concatenation
– Kleene closure operator
– Substitution
Prove
– Homomorphism, inverse homomorphism
this first
– Reversal

48
 Note: s(L) can use
a different alphabet

The Substitution operation

For each a  ∑, then let s(a) be a language

If w=a1a2…an  L, then:
• s(w) = { x1x2 … }  s(L), s.t., xi  s(ai)
Example:
– Let ∑={0,1}
– Let: s(0) = {anbn | n ≥1}, s(1) = {aa,bb}
– If w=01, s(w)=s(0).s(1)
• E.g., s(w) contains a1 b1 aa, a1 b1bb,
a2 b2 aa, a2 b2bb,
… and so on.

49
 CFLs are closed under Substitution

IF L is a CFL and a substititution defined on L, s(L), is s.t.,

s(a) is a CFL for every symbol a, THEN:
– s(L) is also a CFL

What is s(L)?
L s(L)
w1 s(w1) Note: each s(w)
w2 s(L) s(w2) is itself a set of strings
w3 s(w3)
w4 s(w4)
… …
50
 CFLs are closed under Substitution

– G=(V,T,P,S) : CFG for L

– Because every s(a) is a CFL, there is a CFG for each s(a)
• Let Ga = (Va,Ta,Pa,Sa)
– Construct G’=(V’,T’,P’,S) for s(L)
• P’ consists of:
– The productions of P, but with every occurrence of terminal “a” in their
bodies replaced by Sa.
– All productions in any Pa, for any a  ∑
Parse tree for G’: S

Sa1 Sa2 San

…
x1 x2 xn
51
 Substitution of a CFL: example

– Let L = language of binary palindromes s.t., substitutions for 0 and 1

are defined as follows:
• s(0) = {anbn | n ≥1}, s(1) = {xx,yy}
– Prove that s(L) is also a CFL.

CFG for s(0):

CFG for L: CFG for s(1):
S0=> aS0b | ab
S=> 0S0|1S1| S1=> xx | yy

Therefore, CFG for s(L):

S=> S0SS0 | S1 S S1 |
S0=> aS0b | ab
S1=> xx | yy
52
 CFLs are closed under union
Let L1 and L2 be CFLs
To show: L2 U L2 is also a CFL
Let us show by using the result of Substitution
• Make a new language:
– Lnew = {a,b} s.t., s(a) = L1 and s(b) = L2
==> s(Lnew) == same as == L1 U L2
• A more direct, alternative proof
– Let S1 and S2 be the starting variables of the grammars
for L1 and L2
• Then, Snew => S1 | S2

53
 CFLs are closed under concatenation
• Let L1 and L2 be CFLs
Let us show by using the result of Substitution
• Make Lnew= {ab} s.t.,
s(a) = L1 and s(b)= L2
==> L1 L2 = s(Lnew)

• A proof without using substitution?

54
 CFLs are closed under Kleene Closure

• Let L be a CFL

– Let Lnew = {a}* and s(a) = L1

– Then, L* = s(Lnew)

55
 CFLs are closed under Reversal

• Let L be a CFL, with grammar G=(V,T,P,S)

• For LR, construct GR=(V,T,PR,S) s.t.,
– If A==>  is in P, then:
• A==> R is in PR

• (that is, reverse every production)

56
 CFLs are not closed under Intersection
• Existential proof:
– L1 = {0n1n2i | n≥1,i≥1}
– L2 = {0i1n2n | n≥1,i≥1}
• Both L1 and L1 are CFLs
– Grammars?
• But L1  L2 cannot be a CFL
– Why?
• We have an example, where intersection is not
closed.
• Therefore, CFLs are not closed under
intersection
57
 CFLs are not closed under complementation

• Follows from the fact that CFLs are not closed under
intersection

• L1  L2 = L1 U L2

Logic: if CFLs were to be closed under complementation

 the whole right hand side becomes a CFL (because
CFL is closed for union)
 the left hand side (intersection) is also a CFL
 but we just showed CFLs are
NOT closed under intersection!
 CFLs cannot be closed under complementation.

58
 Department of Computer and Science Engineering (CSE)

Outcomes

• Knowledge of Normal forms

• Knowledge of CNF,GNF
• CFL Closure Properties
 Department of Computer and Science Engineering (CSE)

FAQ

1. Convert the following grammar G in GNF form S / ABb /

a , A / aaA / B , B / bAb
2. Show that id+id*id can be generated by two distinct
leftmost derivation in the grammar E*E+E/ E*E/ (E)/id
3. Let G ={{s , c }, {a , b }, P , s } where P consists of S / aca , C
/ aca / b ,Find L(G).
 Department of Computer and Science Engineering (CSE)

References
• Hopcroft J.E. and Ullman J.D., “Introduction to Automata
Theory Languages and Computation”, Narosa
Publications.
• Sipser,” Theory Of Computation, Cengage Learning.
• https://youtu.be/qmX7DJ9j1-A
• http://studentsfocus.com/
THANK YOU

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

UNIVERSITY INSTITUTE OF
ENGINEERING
COMPUTER SCIENCE
ENGINEERING
Bachelor of Engineering
Theory of Computation (CST-353)

Topic: Undecidability
DISCOVER . LEARN .
University Institute of Engineering (UIE) EMPOWER
 Department of Computer and Science Engineering (CSE)

Learning Objectives & Outcomes

Objective:
• To understand the concept of Context Sensitive
Language.

Outcome:
Student will understand the
• To know about Context sensitive language and
grammar.
• Relation between languages of classes.

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Context-sensitive Grammar (CSG) and Language (CSL)

Context-Sensitive Grammar –
A Context-sensitive grammar is an Unrestricted grammar in
which all the productions are of form –

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

• Context-sensitive grammar has 4-tuples. G = {N, Σ, P, S},

Where
N = Set of non-terminal symbols
Σ = Set of terminal symbols
S = Start symbol of the production
P = Finite set of productions

All rules in P are of the form α1 A α2 –> α1 β α2

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Example –
Consider the following CSG.
S → abc/aAbc
Ab → bA
Ac → Bbcc
bB → Bb
aB → aa/aaA

What is the language generated by this grammar?

Solution:
S → aAbc
→ abAc
→ abBbcc
→ aBbbcc
→ aaAbbcc
→ aabAbcc
→ aabbAcc
→ aabbBbccc
→ aabBbbccc
→ aaBbbbccc
→ aaabbbccc
The language generated by this grammar is {anbncn | n≥1}.

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Relation between languages of classes

In order to understand the relationship between the
grammar and language let us understand what is language
generated by grammar.

Language generated by grammar

• The grammar is S-> aSb| epsilon.
• In this grammar, by using S-> epsilon , we can generate
epsilon.
• Therefore, epsilon is part of L(G).
• Similarly, by using S=>aSb=>ab, ab is generated.
• Similarly, aabb can also be generated.

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

• Therefore, the result is as follows −

• L(G) = {anbn, n>0}
• In language L(G) which is discussed above, the condition
n =0 is taken to accept the epsilon.
• Consider the grammar given below
• S -> aSa | bSb | a |b
• Now, let us find out what language is generated by the
above grammar over the alphabet {a,b}.

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

• Explanation
• Using S->a and S->b,
• a and b can be generated.
• Similarly by using S->aSa
• ->aba
• aba can be generated.
• Other strings which can be generated from grammar are
as follows −
• a,b,aba,bab,aaa,bbb,ababa,.....
• Therefore, the language generated for the given grammar
over the alphabet {a,b} is the set of all odd length
palindromes.

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Outcomes
• To know about Context sensitive language and
grammar.
• Relation between languages of classes.

9
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

FAQ
1. Convert the given expression to PDA
I --> a | b | Ia | Ib | I0 | I1
2. Design a PDA for accepting a language {L= anbn | n >=1}
3. What is the language generated by the grammar
G=(V,T,P,S) where P={S->aSb, S->ab}?

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

REFERENCES

• Sipser,” Theory Of Computation, Cengage Learning.

• Daniel I.A. Cohen, “Introduction to computer Theory”, John
Wiley.

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

UNIVERSITY INSTITUTE OF
ENGINEERING
COMPUTER SCIENCE
ENGINEERING
Bachelor of Engineering
Theory of Computation (CST-353)

Topic: PDA DISCOVER . LEARN .

University Institute of Engineering (UIE) EMPOWER
 Department of Computer and Science Engineering (CSE)

Learning Objectives & Outcomes

Objective:
• To understand the concept of Push Down Automata.

Outcome:
• Student will understand the
 Push down automata
 Applications of Push Down Automata

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

PDA - the automata for CFLs

• What is?
– FA to Reg Lang, PDA is to CFL
• PDA == [  -NFA + “a stack” ]
• Why a stack?

Input -NFA Accept/reject

string

A stack filled with “stack symbols”

3
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Pushdown Automata - Definition

• A PDA P := ( Q,∑,, δ,q0,Z0,F ):

– Q: states of the -NFA
– ∑: input alphabet
– : stack symbols
– δ: transition function
– q0: start state
– Z0: Initial stack top symbol
– F: Final/accepting states

4
University Institute of Engineering (UIE)
 old state input symb. Stack top new state(s) new Stack top(s)

δ : Q x ∑ x  => Q x 
δ : The Transition Function

δ(q,a,X) = {(p,Y), …}
1. state transition from q to p
2. a is the next input symbol a X Y
3. X is the current stack top symbol q p
4. Y is the replacement for X;
it is in * (a string of stack Y=? Action
symbols)
i. Set Y =  for: Pop(X) i) Y= Pop(X)
ii. If Y=X: ii) Y=X Pop(X)
stack top is unchanged Push(X)
iii. If Y=Z1Z2…Zk: X is popped
and is replaced by Y iii) Y=Z1Z2..Zk Pop(X)
in Push(Zk)
reverse order (i.e., Z1 will be Push(Zk-1)
the …
Push(Z2)
new stack top) Push(Z1)

University Institute of Engineering (UIE) 5

 Example
Example : Define the pushdown automata for language {anbn | n > 0}
Solution : M = where Q = { q0, q1 } and Σ = { a, b } and Γ = { A, Z }
and &delta is given by :

University Institute of Engineering (UIE) 6

 Department of Computer and Science Engineering (CSE)

PDA’s Instantaneous Description (ID)

A PDA has a configuration at any given instance: (q,w,y)
– q - current state
– w - remainder of the input (i.e., unconsumed part)
– y - current stack contents as a string from top to bottom of
stack
If δ(q,a, X)={(p, A)} is a transition, then the following are also true:
– (q, a, X ) |--- (p,,A)
– (q, aw, XB ) |--- (p,w,AB)
|--- sign is called a “turnstile notation” and represents one
move
|---* sign represents a sequence of moves

7
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

How does the PDA for Lwwr work on input “1111”?

All moves made by the non-deterministic PDA

(q0,1111,Z0)

(q1,1111,Z0) Path dies…

(q0,111,1Z0)

(q0,11,11Z0) (q1,111,1Z0) Path dies…

(q0,1,111Z0) (q1,11,11Z0)

(q0,,1111Z0) (q1,1,111Z0) (q1,1,1Z0) Acceptance by

final state:

(q1, ,1111Z0) (q1, ,11Z0) (q1, ,Z0) = empty input

Path dies… AND
Path dies…
(q2, ,Z0) 8final state
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

There are two types of PDAs that one can design:

those that accept by final state or by empty stack
Acceptance by…

• PDAs that accept by final state:

– For a PDA P, the language accepted by P, denoted by L(P)
by final state, is:
• {w | (q0,w,Z0) |---* (q,, A) }, s.t., q  F Checklist:
- input exhausted?
- in a final state?

• PDAs that accept by empty stack:

– For a PDA P, the language accepted by P, denoted by N(P)
by empty stack, is:
• {w | (q0,w,Z0) |---* (q, , ) }, for any q  Q. Checklist:
- input exhausted?
Q) Does a PDA that accepts by empty stack - is the stack empty?
need any final state specified in the design?
10
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Example: L of balanced parenthesis

An equivalent PDA that

PDA that accepts by final state accepts by empty stack
(,Z0 / ( Z0
PF: (,Z0 / ( Z0
PN: (, ( / ( (
), ( / 
(,( / ( (
,Z0 / 
), ( / 

start ,Z0/ Z0 start

q0 q1 q0
,Z0/ Z0 ,Z0/ Z0

How will these two PDAs work on the input: ( ( ( ) ) ( ) ) ( )

11
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

PDAs accepting by final state and empty

stack are equivalent
• PF <= PDA accepting by final state
– PF = (QF,∑, , δF,q0,Z0,F)
• PN <= PDA accepting by empty stack
– PN = (QN,∑, , δN,q0,Z0)
• Theorem:
– (PN==> PF) For every PN, there exists a PF s.t. L(PF)=L(PN)

– (PF==> PN) For every PF, there exists a PN s.t. L(PF)=L(PN)

13
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

How to convert an empty stack PDA into a final state PDA?

PN==> PF construction
• Whenever PN’s stack becomes empty, make PF go to a
final state without consuming any addition symbol
• To detect empty stack in PN: PF pushes a new stack
symbol X0 (not in  of PN) initially before simultating PN

PF = (QN U {p0,pf}, ∑,  U {X0}, δF, p0, X0, {pf})

PF: PN: , X0/ X0

, X0/Z0X0 , X0/ X0
New
p0 q0 , X0/ X0 pf
start
…
 , X0 / X 0
, X0/ X0

14
University Institute of Engineering (UIE)
 Example: Matching parenthesis “(” “)”

PN: ( {q0}, {(,)}, {Z0,Z1}, δN, q0, Z0 ) Pf: ( {p0,q0 ,pf}, {(,)}, {X0,Z0,Z1}, δf, p0, X0 , pf)

δN: δN(q0,(,Z0) = { (q0,Z1Z0) } δf: δf(p0, ,X0) = { (q0,Z0) }

δN(q0,(,Z1) = { (q0, Z1Z1) } δf(q0,(,Z0) = { (q0,Z1 Z0) }
δf(q0,(,Z1) = { (q0, Z1Z1) }
δN(q0,),Z1) = { (q0, ) }
δf(q0,),Z1) = { (q0, ) }
δN(q0, ,Z0) = { (q0, ) } δf(q0, ,Z0) = { (q0, ) }
δf(p0, ,X0) = { (pf, X0 ) }

(,Z0 /Z1Z0 (,Z0/Z1Z0

(,Z1 /Z1Z1 (,Z1/Z1Z1
),Z1 /  ),Z1/ 
,Z0 /   ,Z0/ 

start
q0
start
,X0/Z0X0 ,X0/ X0
p0 q0 pf

Accept by empty stack Accept by final state

University Institute of Engineering (UIE) 15
 Department of Computer and Science Engineering (CSE)

How to convert an final state PDA into an empty stack PDA?

PF==> PN construction
• Main idea:
– Whenever PF reaches a final state, just make an  -transition into a new
end state, clear out the stack and accept
– Danger: What if PF design is such that it clears the stack midway without
entering a final state?
 to address this, add a new start symbol X0 (not in  of PF)
PN = (Q U {p0,pe}, ∑,  U {X0}, δN, p0, X0)

PN:
, X0/Z0X0 , any/  , any/ 
New
p0 q0 , any/  pe
start
…
, any/ 
PF
16
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

CFGs == PDAs ==> CFLs

PDA by PDA by
≡
final state empty stack

?
CFG

17
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Example 1

Let Lwwr = {wwR | w is in (0+1)*}

• CFG for Lwwr : S==> 0S0 | 1S1 | 
• PDA for Lwwr :
• P := ( Q,∑, , δ,q0,Z0,F )

= ( {q0, q1, q2},{0,1},{0,1,Z0},δ,q0,Z0,{q2})

18
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)
Initial state of the PDA:

PDA for Lwwr Stack q0

1. δ(q0,0, Z0)={(q0,0Z0)} top Z0
2. δ(q0,1, Z0)={(q0,1Z0)}
First symbol push on stack
3. δ(q0,0, 0)={(q0,00)}
4. δ(q0,0, 1)={(q0,01)}
5. δ(q0,1, 0)={(q0,10)}
6. δ(q0,1, 1)={(q0,11)} Grow the stack by pushing
new symbols on top of old
7. δ(q0, , 0)={(q1, 0)} (w-part)
8. δ(q0, , 1)={(q1, 1)}
9. δ(q0, , Z0)={(q1, Z0)} Switch to popping mode
(boundary between w and wR)
10. δ(q1,0, 0)={(q1, )}
11. δ(q1,1, 1)={(q1, )}
Shrink the stack by popping matching
symbols (wR-part)
12. δ(q1, , Z0)={(q2, Z0)}

19
University Institute of Engineering (UIE)
 PDA as a state diagram

δ(qi,a, X)={(qj,Y)}

Next Current Stack

input stack Top
Current symbol top Replacement
state (w/ string Y)

a, X / Y Next
qi qj state

University Institute of Engineering (UIE) 20

 PDA for Lwwr: Transition Diagram

Grow stack ∑ = {0, 1}

0, Z0/0Z0 = {Z0, 0, 1}
1, Z0/1Z0 Pop stack for Q = {q0,q1,q2}
0, 0/00
0, 1/01
matching symbols
1, 0/10 0, 0/ 
1, 1/11 1, 1/ 

, Z0/Z0 q0 q1 q2
, Z0/Z0 , Z0/Z0
, 0/0
, 1/1 Go to acceptance
Switch to
popping mode
This would be a non-deterministic PDA

University Institute of Engineering (UIE) 21

 Example 2: language of balanced
paranthesis

Pop stack for ∑ = { (, ) }

matching symbols = {Z0, ( }
Grow stack
Q = {q0,q1,q2}
(, Z0 / ( Z0
(, ( / ( ( ), ( / 

q0 q1 q2
, Z0 / Z0 ), ( /  , Z0 / Z0
, Z0 / Z0 Go to acceptance (by final state)
when you see the stack bottom symbol
Switch to
(, ( / ( (
popping mode
(, Z0 / ( Z0 To allow adjacent
blocks of nested paranthesis

University Institute of Engineering (UIE) 22

Example 2: language of balanced
paranthesis (another design)

∑ = { (, ) }
(,Z0 / ( Z0 = {Z0, ( }
(,( / ( (
), ( / 
Q = {q0,q1}

start ,Z0/ Z0
q0 q1
,Z0/ Z0

University Institute of Engineering (UIE) 23

 Example 3
Construct a PDA that accepts the language L over {0, 1} by empty
stack which accepts all the string of 0's and 1's in which a number of
0's are twice of number of 1's.
There are two parts for designing this PDA:
If 1 comes before any 0's
If 0 comes before any 1's.

We are going to design the first part i.e. 1 comes before 0's. The
logic is that read single 1 and push two 1's onto the stack.
Thereafter on reading two 0's, POP two 1's from the stack. The δ
can be
δ(q0, 1, Z) = (q0, 11, Z) Here Z represents that stack is empty
δ(q0, 0, 1) = (q0, ε)
Now, consider the second part i.e. if 0 comes before 1's. The logic
is that read first 0, push it onto the stack and change state from q0
to q1. [Note that state q1 indicates that first 0 is read and still
second 0 has yet to read].
University Institute of Engineering (UIE) 24
 Continue..
Being in q1, if 1 is encountered then POP 0. Being in q1, if 0 is
read then simply read that second 0 and move ahead. The δ will
be:
δ(q0, 0, Z) = (q1, 0Z)
δ(q1, 0, 0) = (q1, 0)
δ(q1, 0, Z) = (q0, ε) (indicate that one 0 and one 1 is already r
ead, so simply read the second 0)
δ(q1, 1, 0) = (q1, ε)
Now, summarize the complete PDA for given L is:
δ(q0, 1, Z) = (q0, 11Z)
δ(q0, 0, 1) = (q1, ε)
δ(q0, 0, Z) = (q1, 0Z)
δ(q1, 0, 0) = (q1, 0)
δ(q1, 0, Z) = (q0, ε)
δ(q0, ε, Z) = (q0, ε) ACCEPT state

University Institute of Engineering (UIE) 25

 Department of Computer and Science Engineering (CSE)

Application of PDA

• For designing the parsing phase of a compiler (Syntax

Analysis).
• For implementation of stack applications.
• For evaluating the arithmetic expressions.
• For solving the Tower of Hanoi Problem.

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Summary

• PDAs for CFLs and CFGs

– Non-deterministic
– Deterministic
• PDA acceptance types
1. By final state
2. By empty stack
• PDA
– IDs, Transition diagram

• Applications of PDA

27
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

FAQ :

1. What are the different ways of language acceptances by

a PDA and define them
2. Convert the given expression to PDA
I --> a | b | Ia | Ib | I0 | I1

3. What is the additional feature a PDA has when compared

with NFA?
4. Design a PDA for accepting a language {L= a^nb^n | n
>=1}

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

REFERENCES :

• https://www.geeksforgeeks.org/introduction-of-
pushdown-automata/
• Martin J.C., “Introduction to Languages and Theory of
Computation”, Tata McGraw-Hill Publishing Company
Limited, 3rd Edition.
• https://www.tutorialspoint.com
• https://en.wikipedia.org

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

UNIVERSITY INSTITUTE OF
ENGINEERING
COMPUTER SCIENCE
ENGINEERING
Bachelor of Engineering
Theory of Computation (CST-353)

Topic: Finite Automata

DISCOVER . LEARN .
University Institute of Engineering (UIE) EMPOWER
 Department of Computer and Science Engineering (CSE)

Learning Objectives & Outcomes

Objective:
• To understand the concept of Finite Automata, properties
and limitation of Finite Automata.

Outcome:
• Student will understand the
 Finite Automata- DFA, NFA

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Finite Automaton (FA)

• Informally, a state diagram that comprehensively
captures all possible states and transitions that a
machine can take while responding to a stream or
sequence of input symbols
• Recognizer for “Regular Languages”

• Deterministic Finite Automata (DFA)

– The machine can exist in only one state at any given time
• Non-deterministic Finite Automata (NFA)
– The machine can exist in multiple states at the same time

3
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Deterministic Finite Automata - Definition

• A Deterministic Finite Automaton (DFA) consists

of:
– Q ==> a finite set of states
– ∑ ==> a finite set of input symbols (alphabet)
– q0 ==> a start state
– F ==> set of final states
– δ ==> a transition function, which is a mapping between
Q x ∑ ==> Q
• A DFA is defined by the 5-tuple:
– {Q, ∑ , q0,F, δ }

4
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

What does a DFA do on reading an input string?

• Input: a word w in ∑*
• Question: Is w acceptable by the DFA?
• Steps:
– Start at the “start state” q0
– For every input symbol in the sequence w do
• Compute the next state from the current state, given the current
input symbol in w and the transition function
– If after all symbols in w are consumed, the current state
is one of the final states (F) then accept w;
– Otherwise, reject w.

5
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Regular Languages

• Let L(A) be a language recognized by a DFA A.

– Then L(A) is called a “Regular Language”.

• Locate regular languages in the Chomsky Hierarchy

6
University Institute of Engineering (UIE)
 The Chomsky Hierachy
• A containment hierarchy of classes of formal languages

Regular Context-
(DFA) Context- Recursively-
free
sensitive enumerable
(PDA)
(LBA) (TM)

University Institute of Engineering (UIE) 7

 Department of Computer and Science Engineering (CSE)

Example #1
• Build a DFA for the following language:
– L = {w | w is a binary string that contains 01 as a substring}
• Steps for building a DFA to recognize L:
– ∑ = {0,1}
– Decide on the states: Q
– Designate start state and final state(s)
– δ: Decide on the transitions:
• Final states == same as “accepting states”
• Other states == same as “non-accepting states”

8
University Institute of Engineering (UIE)
 Regular expression: (0+1)*01(0+1)*
DFA for strings containing 01

• What makes this DFA deterministic? • Q = {q0,q1,q2}

1 0 0,1 • ∑ = {0,1}
• start state = q0
start 0 1
q0 q1 q2 • F = {q2}
Final • Transition table
state symbols
0 1
q0 q1 q0
• What if the language allows
states

q1 q1 q2
empty strings? *q2 q2 q2

University Institute of Engineering (UIE) 9

 Department of Computer and Science Engineering (CSE)

Example #2

Clamping Logic:
– A clamping circuit waits for a ”1” input, and turns on forever.
However, to avoid clamping on spurious noise, we’ll design a DFA
that waits for two consecutive 1s in a row before clamping on.
• Build a DFA for the following language:
L = { w | w is a bit string which contains the
substring 11}
• State Design:
– q0 : start state (initially off), also means the most recent input was
not a 1
– q1: has never seen 11 but the most recent input was a 1
– q2: has seen 11 at least once

10
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Example #3

• Build a DFA for the following language:

L = { w | w is a binary string that has even
number of 1s and even number of 0s}
• ?

11
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Extension of transitions (δ) to Paths (δ)

• δ (q,w) = destination state from state q on input string w

• δ (q,wa) = δ (δ(q,w), a)

– Work out example #3 using the input sequence w=10010, a=1:

• δ (q0,wa) = ?

12
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Language of a DFA
A DFA A accepts string w if there is a path from q0 to an
accepting (or final) state that is labeled by w

• i.e., L(A) = { w | δ(q0,w)  F }

• I.e., L(A) = all strings that lead to a final state from q0

13
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Non-deterministic Finite Automata (NFA)

• A Non-deterministic Finite Automaton (NFA)

– is of course “non-deterministic”
• Implying that the machine can exist in more than one state at
the same time
• Transitions could be non-deterministic

1 qj
qi • Each transition function therefore
…
1 maps to a set of states
qk
14
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Non-deterministic Finite Automata (NFA)

• A Non-deterministic Finite Automaton (NFA)

consists of:
– Q ==> a finite set of states
– ∑ ==> a finite set of input symbols (alphabet)
– q0 ==> a start state
– F ==> set of final states
– δ ==> a transition function, which is a mapping between
Q x ∑ ==> subset of Q
• An NFA is also defined by the 5-tuple:
– {Q, ∑ , q0,F, δ }

15
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

How to use an NFA?

• Input: a word w in ∑*
• Question: Is w acceptable by the NFA?
• Steps:
– Start at the “start state” q0
– For every input symbol in the sequence w do
• Determine all possible next states from all current states, given the
current input symbol in w and the transition function
– If after all symbols in w are consumed and if at least one of the
current states is a final state then accept w;
– Otherwise, reject w.

16
University Institute of Engineering (UIE)
 Regular expression: (0+1)*01(0+1)*
NFA for strings containing 01

Why is this non-deterministic?

• Q = {q0,q1,q2}
0,1 0,1 •  = {0,1}
• start state = q0
start 0 1
q0 q1 q2 • F = {q2}
Final • Transition table
state symbols
0 1
What will happen if at state q1 q0 {q0,q1} {q0}
states

an input of 0 is received? q1 Φ {q2}

*q2 {q2} {q2}

University Institute of Engineering (UIE) 17

 Department of Computer and Science Engineering (CSE)

Transitions into a dead state are implicit

What is a “dead state”?
• A DFA for recognizing the key word “while”

w h i l e
q0 q1 q2 q3 q4 q5

Any other input symbol

qdead
Any symbol

w h i l e
q0 q1 q2 q3 q4 q5

18
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Example #2

• Build an NFA for the following language:

L = { w | w ends in 01}
• ?
• Other examples
– Keyword recognizer (e.g., if, then, else, while, for, include, etc.)
– Strings where the first symbol is present somewhere later on at
least once

19
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Extension of δ to NFA Paths

• Basis: δ (q,) = {q}

• Induction:
– Let δ (q0,w) = {p1,p2…,pk}
– δ (pi,a) = Si for i=1,2...,k

– Then, δ (q0,wa) = S1 U S2 U … U Sk

20
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Language of an NFA

• An NFA accepts w if there exists at least one path from the

start state to an accepting (or final) state that is labeled
by w
• L(N) = { w | δ(q0,w) ∩ F ≠ Φ }

21
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Advantages & Caveats for NFA

• Great for modeling regular expressions
– String processing - e.g., grep, lexical analyzer
• Could a non-deterministic state machine be
implemented in practice?
– A parallel computer could exist in multiple “states” at the
same time
– Probabilistic models could be viewed as extensions of
non-deterministic state machines
(e.g., toss of a coin, a roll of dice)

22
University Institute of Engineering (UIE)
DFAs and NFAs are equivalent in their power to capture langauges !!

Differences: DFA vs. NFA

• NFA
• DFA 1. Some transitions could be non-
1. All transitions are deterministic
deterministic – A transition could lead to a
– Each transition leads to exactly subset of states
one state 2. Not all symbol transitions need
2. For each state, transition on all to be defined explicitly (if
possible symbols (alphabet) undefined will go to a dead
should be defined state – this is just a design
3. Accepts input if the last state is convenience, not to be confused
in F with “non-determinism”)
4. Sometimes harder to construct 3. Accepts input if one of the last
because of the number of states states is in F
5. Practical implementation is 4. Generally easier than a DFA to
feasible construct
5. Practical implementation has to
be deterministic (convert to
DFA) or in the form of
parallelism

University Institute of Engineering (UIE) 23

 Department of Computer and Science Engineering (CSE)

Summary
• DFA
– Definition
– Transition diagrams & tables
• Regular language
• NFA
– Definition
– Transition diagrams & tables
• DFA vs. NFA
• NFA to DFA conversion using subset construction

24
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

FAQ
1. How a Non deterministic finite state automaton (NFA)
differs from a Deterministic finite state automaton
(DFA).
2. Write RE which denotes the language L over the set =
{a,b} such that all the strings do not contain the substring
ab.
3. Give a regular expression for the set of all strings having
odd number of 1’s
4. Prove that 12 +22+32+……+ n2= ∑n2 =
(n(n+1)(2n+1))/6.

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

References
• Https://en.wikipedia.org/wiki/Finite-state_machine
• https://www.safaribooksonline.com
• http://studentsfocus.com/
• http://www.francisxavier.ac.in/

University Institute of Engineering (UIE)

THANK YOU

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

UNIVERSITY INSTITUTE OF
ENGINEERING
COMPUTER SCIENCE
ENGINEERING
Bachelor of Engineering
Theory of Computation (CST-353)

Topic: Finite Automata

DISCOVER . LEARN .
University Institute of Engineering (UIE) EMPOWER
 Department of Computer and Science Engineering (CSE)

Learning Objectives & Outcomes

Objective:
• To understand the concept of Finite Automata, properties
and limitation of Finite Automata.

Outcome:
• Student will understand the
 Finite Automata- DFA, NFA
 Properties of FA

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Finite Automaton (FA)

Finite automata is an abstract computing device. It is a
mathematical model of a system with discrete inputs,
outputs, states and a set of transitions from state to state
that occurs on input symbols from the alphabet Σ.

Finite Automata Representation

The finite automata can be represented in three ways, as
given below −
• Graphical (Transition diagram)
• Tabular (Transition table)
• Mathematical (Transition function)

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Formal definition of Finite Automata

• Finite automata is defined as a 5-tuples

M=(Q, Σ, δ,q0,F)
Where,
• Q: Finite set called states.
• Σ: Finite set called alphabets.
• δ: Q × Σ → Q is the transition function.
• q0 ∈ Q is the start or initial state.
• F: Final or accept state.

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Finite Automata can be represented as follows −

• Transition diagram
• Transition table
• Transition function
Transition Diagram
• It is a directed graph associated with the vertices of the
graph corresponding to the state of finite automata.
• An example of transition diagram is given below −

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Transition table

It is basically a tabular representation of the transition

function that takes two arguments (a state & a symbol) and
returns a value (the ‘next state’).
• δ:Q×Σ→Q
• In transition table, the following factors are considered −
• Rows correspond to state.
• Column corresponds to the input symbol.
• Entries correspond to the next state.
• The start state is marked with ->.
• The accept state marked with *.

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

An example of transition table is as follows −

The transition table is as follows −

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Transition function

The transition function is denoted by δ. The two parameters

mentioned below are the passes to this transition function.
• Current state
• Input symbol
• The transition function returns a state which can be
called as the next state.
δ (current_state, current_input_symbol) = next_state
• For example, δ(q0,a)=q1

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Properties and Limitations of Finite State Machines

There are many properties available when defining an FSM. These
properties are available at the various levels of the FSM hierarchy.

• FSM Level Properties

• State Properties

• Resolution Action Properties

• Transition properties

• Event Type Properties

• Event Context Properties

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Disadvantages of Finite State Machine

The disadvantages of the finite state machine include the

following
• The expected character of deterministic finite state
machines can be not needed in some areas like computer
games
• The implementation of huge systems using FSM is hard
for managing without any idea of design.
• Not applicable for all domains
• The orders of state conversions are inflexible.

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

References
• Https://en.wikipedia.org/wiki/Finite-state_machine
• https://www.safaribooksonline.com
• http://studentsfocus.com/
• http://www.francisxavier.ac.in/

University Institute of Engineering (UIE)

THANK YOU

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

UNIVERSITY INSTITUTE OF
ENGINEERING
COMPUTER SCIENCE
ENGINEERING
Bachelor of Engineering
Theory of Computation (CST-353)

Topic: Kleen’s Theorem

DISCOVER . LEARN .
University Institute of Engineering (UIE) EMPOWER
 Department of Computer and Science Engineering (CSE)

Learning Objectives & Outcomes

Objective:
• To understand the concept of Kleene’s Theorem.

Outcome:
• Student will understand the
 Kleene’s Theorem

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Kleene’s Theorem
Theorem 1 (Part 1 of Kleene's theorem): Any regular
language is accepted by a finite automaton.

Proof: This is going to be proven by (general) induction

following the recursive definition of regular language.

3
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Inductive Step

• Inductive Step: We are going to show that for any

languages L1 and L2 if they are accepted by FAs, then
L1∪ L2 , L1L2 and L1* are accepted by FAs. Since any
regular language is obtained from { ʌ } and { a } for any
symbol a in Σ by using union, concatenation and Kleene
star operations, that together with the Basis Step would
prove the theorem.

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Inductive Step (Continue..)

• Suppose that L1 and L2 are accepted by FAs M1 = < Q1 , Σ , q1,0 , δ 1 ,
A1 > and M2 = < Q2 , Σ , q2,0 , δ 2 , A2 > , respectively. We assume that
Q1 Q2 = Ø without loss of generality since states can be renamed if
necessary.
Then L1∪ L2 , L1L2 and L1* are accepted by the FAs Mu = < Qu , Σ ,
qu,0 , δ u , Au > , Mc = < Qc , Σ , qc,0 , δ c , Ac > and Mk = < Q2 , Σ , qk,0 , δ k ,
Ak > , respectively, which are given below.
Mu = < Qu , Σ , qu,0 , δ u , Au > :
Qu = Q1 ∪ Q2 ∪ { qu,0 } , where qu,0 is a state which is neither in Q1 nor
in Q2 .
δ u = δ 1 ∪ δ 2 ∪ { (qu,0, ʌ , { q1,0 , q2,0 } ) } , that is δ u(qu,0, ʌ ) = { q1,0 ,
q2,0 } . Note that δ u(qu,0, a ) = Ø for all a in Σ .
Au = A1 ∪ A2

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Inductive Step (Continue..)

Mc = < Qc , Σ , qc,0 , δ c , Ac > :

Qc = Q1 ∪ Q2
qc,0 = q1,0
δ c = δ 1 ∪ δ 2 ∪ { (q, ʌ , { q2,0 } ) | q ∈ A1 }
Ac = A2

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Inductive Step (Continue..)

Mk = < Qk , Σ , qk,0 , δ k , Ak > :

Qk = Q1 ∪ { qk,0 } , where qk,0 is a state which is not in Q1 .

δ k = δ 1 ∪ { (qk,0, ʌ , { q1,0 } ) } ∪ { (q, ʌ , { qk,0 } ) | q ∈ A1 }
Ak = { qk,0 }

These NFA- ʌ s are illustrated below.

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

It can be proven, though we omit proofs, that these NFA- ʌ s , Mu,

Mc and Mk , in fact accept L1∪ L2 , L1L2 and L1*, respectively.
End of Proof

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Kleene’s Theorem Part-2

Kleene’s Theorem Part-2

In this section we prove that if L is accepted by a finite

automaton, then L is regular. The proof will provide an
algorithm for starting with an FA that accepts L and finding
a regular expression that describes L.

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Summary

• Kleene’s Theorem
– Part 1
– Part 2

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

References

• Martin J.C., “Introduction to Languages and Theory of

Computation”, Tata McGraw-Hill Publishing Company
Limited, 3rd Edition
• https://www.geeksforgeeks.org/kleenes-theorem-in-toc-
part-1/
• https://www.cs.odu.edu/~toida/nerzic/390teched/regu
lar/fa/kleene-
1.html#:~:text=It%20states%20that%20any%20regular,
by%20an%20FA%20is%20regular

University Institute of Engineering (UIE)

THANK YOU

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

UNIVERSITY INSTITUTE OF
ENGINEERING
COMPUTER SCIENCE
ENGINEERING
Bachelor of Engineering
Theory of Computation (CST-353)

Topic: Context Free Language

DISCOVER . LEARN .
EMPOWER
 Department of Computer and Science Engineering (CSE)

Learning Objectives & Outcomes

Objective:
• To understand the concept of ambiguity and Types of
grammar.

Outcome:
• Student will understand the
 Ambiguity
 Ambiguous and unambiguous grammar
 Department of Computer and Science Engineering (CSE)

Types of Grammar

On the basis of number of derivation trees, grammars are

classified as-
 Department of Computer and Science Engineering (CSE)

Ambiguous Grammar

• A grammar is said to ambiguous if for any string

generated by it, it produces more than one-

• Parse tree
• Or derivation tree
• Or syntax tree
• Or leftmost derivation
• Or rightmost derivation
 Department of Computer and Science Engineering (CSE)

• Example-
• Consider the following grammar-
• E → E + E / E x E / id

Since two parse trees exist for string w, therefore the grammar is
ambiguous.
Ambiguity in CFGs and CFLs

6
 Ambiguity in CFGs

• A CFG is said to be ambiguous if there exists a string

which has more than one left-most derivation

Example:
S ==> AS |  LM derivation #1: LM derivation #2:
A ==> A1 | 0A1 | 01 S => AS S => AS
=> 0A1S => A1S
=>0A11S => 0A11S
Input string: 00111 => 00111S => 00111S
Can be derived in two ways => 00111 => 00111

7
 Why does ambiguity matter?

Values are
E ==> E + E | E * E | (E) | a | b | c | 0 | 1 different !!!
string = a * b + c
E
• LM derivation #1:
•E => E + E => E * E + E E + E (a*b)+c
==>* a * b + c
E * E c

a b
E
• LM derivation #2
•E => E * E => a * E => E * E a*(b+c)
a * E + E ==>* a * b + c
a E + E
The calculated value depends on which
of the two parse trees is actually used. b c

8
 Removing Ambiguity in Expression Evaluations

• It MAY be possible to remove ambiguity for some

CFLs
– E.g.,, in a CFG for expression evaluation by imposing
rules & restrictions such as precedence
– This would imply rewrite of the grammar

• Precedence: (), * , + Modified unambiguous version:

E => E + T | T
Ambiguous version: T => T * F | F
F => I | (E)
E ==> E + E | E * E | (E) | a | b | c | 0 | 1 I => a | b | c | 0 | 1

9
 Inherently Ambiguous CFLs

• However, for some languages, it may not be

possible to remove ambiguity

• A CFL is said to be inherently ambiguous if every

CFG that describes it is ambiguous
Example:
– L = { anbncmdm | n,m≥ 1} U {anbmcmdn | n,m≥ 1}
– L is inherently ambiguous
– Why?
Input string: anbncndn

10
 Department of Computer and Science Engineering (CSE)

Unambiguous Grammar-

• A grammar is said to unambiguous if for every string

generated by it, it produces exactly one-

• Parse tree
• Or derivation tree
• Or syntax tree
• Or leftmost derivation
• Or rightmost derivation
 Department of Computer and Science Engineering (CSE)

Example 1:
• Consider a grammar G is given as follows:
• S → AB | aaB
• A → a | Aa
• B→b
Determine whether the grammar G is ambiguous or not. If G
is ambiguous, construct an unambiguous grammar
equivalent to G.
Department of Computer and Science Engineering (CSE)

Solution:
Let us derive the string "aab"
 Department of Computer and Science Engineering (CSE)

• As there are two different parse tree for deriving the

same string, the given grammar is ambiguous.

Unambiguous grammar will be:

• S → AB
• A → Aa | a
• B→b
 Department of Computer and Science Engineering (CSE)

• Example-
• Consider the following grammar-

• E→E+T/T
• T→TxF/F
• F → id

This grammar is an example of unambiguous grammar.

 Department of Computer and Science Engineering (CSE)

Outcomes

• Knowledge of ambiguity.
• Knowledge of ambiguous and unambiguous grammar.
 Department of Computer and Science Engineering (CSE)

FAQ :

1. Find L(G),where P consists of S ® asb , s ® ab .

2. What are the closure properties of context- free
Language?
3. Write CFG for L(G) = {a m b n c p / m + n = p , p ³ 1}.
 Department of Computer and Science Engineering (CSE)

REFERENCES :

• Martin J.C., “Introduction to Languages and Theory of

Computation”, Tata McGraw-Hill Publising Company
Limited, 3rd Edition.
• Daniel I.A. Cohen, “Introduction to computer Theory”, John
Wiley.
• K.L.P MISHRA, “Introduction to computer Theory” .
• https://youtu.be/NNSK4570Kho
• https://youtu.be/A_Zj-WaA2Lo
THANK YOU

University Institute of Engineering (UIE)

 Department of Computer Science and Engineering (CSE)

UNIVERSITY INSTITUTE OF
ENGINEERING
COMPUTER SCIENCE
ENGINEERING
Bachelor of Engineering
Theory of Computation (CST-353)

Topic: Recursively Enumerable

DISCOVER . LEARN .
University Institute of Engineering (UIE) EMPOWER
 Department of Computer Science and Engineering (CSE)

Learning Objectives & Outcomes

Objective:
• To understand the concept of Recursively Enumerable
and Recursive Languages, Unrestricted Grammars and
Chomsky hierarchies of grammars.

Outcome:
• Student will understand the
 Recursively Enumerable and Recursive Languages,
 Unrestricted Grammars
 Chomsky hierarchies of grammars.

University Institute of Engineering (UIE)

 Department of Computer Science and Engineering (CSE)

Recursive Enumerable (RE) or Type -0 Language

• RE languages or type-0 languages are generated by type-

0 grammars. An RE language can be accepted or
recognized by Turing machine which means it will enter
into final state for the strings of language and may or may
not enter into rejecting state for the strings which are not
part of the language. It means TM can loop forever for the
strings which are not a part of the language. RE
languages are also called as Turing recognizable
languages.

University Institute of Engineering (UIE)

 Department of Computer Science and Engineering (CSE)

Recursive Language (REC)

• A recursive language (subset of RE) can be decided by

Turing machine which means it will enter into final state
for the strings of language and rejecting state for the
strings which are not part of the language. e.g.; L=
{anbncn|n>=1} is recursive because we can construct a
turing machine which will move to final state if the string
is of the form anbncn else move to non-final state.
• So the TM will always halt in this case. REC languages are
also called as Turing decidable languages.

University Institute of Engineering (UIE)

 Department of Computer Science and Engineering (CSE)

• Lemma: Ld is not recursively enumerable: [No TM!!!]

• Proof: (by contradiction)

Suppose that Ld were recursively enumerable. In other words, that there existed a TM M such that:

Ld = L(M) (2)

Now suppose that wj is a string encoding of M. (3)

Case 1) wj is in Ld (4)

By definition of Ld given in (1), either wj does not encode a TM, or wj does encode a TM, call it M, and wj is not in
L(M). But we know that wj encodes a TM (3: that’s where it came from). Therefore:

wj is not in L(M) (5)

But then (2) and (5) imply that wj is not in Ld contradicting (4).

Case 2) wj is not in Ld (6)

By definition of Ld given in (1), wj encodes a TM, call it M, and:

wj is in L(M) (7)

But then (2) and (7) imply that wj is in Ld contradicting (6).

Since both case 1) and case 2) lead to a contradiction, no TM M such that Ld = L(M) can exist. Therefore Ld is not
recursively enumerable. •
5
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

• Note:

= {x | x is in {0, 1}*, x encodes a TM, call it M, and x is in L(M)}

Ld

• Corollary: is not recursive.

Ld
• Proof: If were recursive, then Ld would be recursive, and therefore
Ld
recursively enumerable, a contradiction.•

6
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)
• Theorem: Lu is not recursive.

• Proof: (by contradiction)

Suppose that Lu is recursive. Recall that:

Lu = {x | x is in {0, 1}* and x = <M,w> where M is a TM encoding and w is in L(M)}

Suppose that Lu = L(M’) where M’ is a TM that always halts. Construct an algorithm (i.e., a TM
that always halts)Ldfor as follows:

M’’: for Ld-bar

<M,w>
Yes Yes Yes
(i.e., <w,w>)
Let M be the TM M’:
w that w encodes.
Is w a TM? UTM for Lu No No
No

Suppose that M’ always halts and Lu = L(M’). It follows that:

– M’’ always halts
– L(M’’) = Ld

Ld would therefore be recursive, a contradiction.•

7
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

L_u is recursively enumerable

(you may ignore this slide, for now)

Input the string

Decode the TM prefix, if it doesn't have one then the string is not in Lu
Otherwise run/simulate the encoded TM on the suffix
If it terminates and accepts then the original string is in Lu.

If a given string is in Lu, then the above algorithm will correctly determine that, halt and say
yes.

If the given string is not in Lu, then there are three cases:
1) the string doesn't have a TM as a prefix. In this case the above algo correctly detects this
fact, and reports the string is not in Lu.
2) the string has a TM prefix, and the TM halts and rejects on the suffix. In this case the above
algo correctly reports the string is not in Lu.
3) the string has a TM prefix, and the TM goes into an infinite loop on the suffix. In this case
the above algo also goes into an infinite loop, but that’s ok since the string as a whole is not in
Lu anyway, and we are just trying to show there exists a TM for only accepting strings in Lu.

-- due to Dr. Bernhard (edited by me)

University Institute of Engineering (UIE) 8

 Department of Computer Science and Engineering (CSE)

• The over-all logic of the proof is as follows:

1. If Lu were recursive, then so is

2. is not recursive
Ld
3. It follows that Lu is not recursive.
Ld
The second point was established by the corollary.
The first point was established by the theorem on the preceding slide.
This type of proof is commonly referred to as a reduction. Specifically, the problem
of recognizing was reduced to the problem of recognizing Lu

Ld

9
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

• Define another language Lh:

Lh = {x | x is in {0, 1}* and x = <M,w> where M is a TM encoding and M halts on w}

Note that Lh is P2 expressed as a language:

(P2):
Input: Turing machine M with input alphabet Σ and string w in Σ*.
Question: Does M halt on w?

10
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)
• Theorem: Lh is not recursive.

• Proof: (by contradiction)

Suppose that Lh is recursive. Recall that:

Lh = {x | x is in {0, 1}* and x = <M,w> where M is a TM encoding and M halts on w}

and
Lu = {x | x is in {0, 1}* and x = <M,w> where M is a TM encoding and w is in L(M)}

Suppose that Lh = L(M’) where M’ is a TM that always halts. Construct an algorithm (i.e., a TM
that always halts) for Lu as follows:

M’’ : UTM for Lu

Yes start Yes Yes
<M,w> M’: Simulate M
On w No No
does M halt? No

Suppose that M’ always halts and Lh = L(M’). It follows that:

– M’’ always halts
– L(M’’) = Lu

Lu would therefore be recursive, a contradiction.•

11
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

• The over-all logic of the proof is as follows:

1. If Lh is recursive, then so is Lu

2. Lu is not recursive

3. It follows that Lh is not recursive.

The second point was established previously.

The first point was established by the theorem on the preceding slide.
This proof is also a reduction. Specifically, the problem of recognizing Lu
was reduced to the problem of recognizing Lh.

[Lu and Lh both are recursively enumerable: for proof see Dr. Shoaff!]

12
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)
• Define another language Lt:

Lt = {x | x is in {0, 1}*, x encodes a TM M, and M does not contain an infinite loop}

Or equivalently:

Lt = {x | x is in {0, 1}*, x encodes a TM M, and there exists no string w in {0, 1}*

such that M does not terminate on w}

Note that:

Lt
= {x | x is in {0, 1}*, and either x does not encode a TM, or it does encode a TM, call it M,
and there exists a string w in {0, 1}* such that M does not terminate on w}

Note that the above languages correspond to the following problem:

(P0):
Input: Program P.
Question: Does P contain an infinite loop?

13
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

• More examples of non-recursive languages:

Lne = {x | x is a TM M and L(M) is not empty} is r.e. but not

recursive.

Le = {x | x is a TM M and L(M) is empty} is not r.e.

Lr = {x | x is a TM M and L(M) is recursive} is not r.e.

Note that Lr is not the same as Lh = {x | x is a TM M that

always halts}
but Lh is in Lr.

Lnr = {x | x is a TM M and L(M) is not recursive} is not r.e.

University Institute of Engineering (UIE) 14

 Department of Computer Science and Engineering (CSE)

Unrestricted grammars

• Definition: an unrestricted grammar is a 4-

tuple (T,N,P,S), consisting of:
• T = set of terminals (the legal "tokens" of the language)
• N = set of nonterminals (aka variables)
• P = as set of productions, each of the form:
v -> w (what does this mean?)
where v and w are strings consisting of
nonterminals and terminals.
• S = a special nonterminal called the start symbol.

University Institute of Engineering (UIE)

 Department of Computer Science and Engineering (CSE)

Chomsky hierarchies of grammars

• Chomsky Hierarchy represents the class of languages

that are accepted by the different machine. The category
of language in Chomsky's Hierarchy is as given below:
• Type 0 known as Unrestricted Grammar.
• Type 1 known as Context Sensitive Grammar.
• Type 2 known as Context Free Grammar.
• Type 3 Regular Grammar.

University Institute of Engineering (UIE)

 Department of Computer Science and Engineering (CSE)

University Institute of Engineering (UIE)

 Department of Computer Science and Engineering (CSE)

Type 0 Grammar

• Type 0 grammar is known as Unrestricted grammar.

There is no restriction on the grammar rules of these
types of languages. These languages can be efficiently
modeled by Turing machines.
• For example:
• bAa → aa
• S→s

University Institute of Engineering (UIE)

 Department of Computer Science and Engineering (CSE)

Type 1 Grammar
• Type 1 grammar is known as Context Sensitive Grammar. The
context sensitive grammar is used to represent context
sensitive language. The context sensitive grammar follows the
following rules:
• The context sensitive grammar may have more than one
symbol on the left hand side of their production rules.
• The number of symbols on the left-hand side must not exceed
the number of symbols on the right-hand side.
• The rule of the form A → ε is not allowed unless A is a start
symbol. It does not occur on the right-hand side of any rule.
• The Type 1 grammar should be Type 0. In type 1, Production is
in the form of V → T
• Where the count of symbol in V is less than or equal to T.

University Institute of Engineering (UIE)

 Department of Computer Science and Engineering (CSE)

Type 2 Grammar

• Type 2 Grammar is known as Context Free Grammar.

Context free languages are the languages which can be
represented by the context free grammar (CFG). Type 2
should be type 1. The production rule is of the form
• A→α
• Where A is any single non-terminal and is any
combination of terminals and non-terminals.

University Institute of Engineering (UIE)

 Department of Computer Science and Engineering (CSE)

Type 3 Grammar

• Type 3 Grammar is known as Regular Grammar. Regular

languages are those languages which can be described
using regular expressions. These languages can be
modelled by NFA or DFA.
• Type 3 is most restricted form of grammar. The Type 3
grammar should be Type 2 and Type 1. Type 3 should be
in the form of
• V → T*V / T*

University Institute of Engineering (UIE)

 Department of Computer Science and Engineering (CSE)

REFERENCES :
•Hopcroft J.E. and Ullman J.D., “Introduction to Automata Theory
Languages and Computation”, Narosa Publications.
•https://www.geeksforgeeks.org/recursive-and-recursive-
enumerable-languages-in-toc/
•https://www.javatpoint.com/automata-chomsky-hierarchy

University Institute of Engineering (UIE) 22

 Department of Computer Science and Engineering (CSE)

UNIVERSITY INSTITUTE OF
ENGINEERING
COMPUTER SCIENCE
ENGINEERING
Bachelor of Engineering
Theory of Computation (CST-353)

Topic: Examples of Turing Machines

DISCOVER . LEARN .
University Institute of Engineering (UIE) EMPOWER
 Department of Computer Science and Engineering (CSE)

Learning Objectives & Outcomes

Objective:
• To understand the concept of Turing Machines & how to
build TM.

Outcome:
• Student will understand the
 Turing machines

University Institute of Engineering (UIE)

 Department of Computer Science and Engineering (CSE)

Examples: 1

Construct a TM for the language L = {0n1n2n} where n≥1

3
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

Examples: 2

Construct a TM machine for checking the palindrome of the string of

even length.

4
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

Examples: 3

Construct a TM machine for checking the palindrome of the string of

odd length.

5
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

Examples: 4

Construct TM for the addition function for the unary number system.

6
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

Examples: 5

• Construct a TM for subtraction of two unary numbers f(a-b) = c

where a is always greater than b.

7
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

Summary

• Turing machine
– Examples of Turing Machines

University Institute of Engineering (UIE)

 Department of Computer Science and Engineering (CSE)

REFERENCES :
•Hopcroft J.E. and Ullman J.D., “Introduction to Automata Theory
Languages and Computation”, Narosa Publications.
•https://www.javatpoint.com/examples-of-turing-machine

University Institute of Engineering (UIE) 9

 Department of Computer Science and Engineering (CSE)

UNIVERSITY INSTITUTE OF
ENGINEERING
COMPUTER SCIENCE
ENGINEERING
Bachelor of Engineering
Theory of Computation (CST-353)

Topic: Turing Machines

DISCOVER . LEARN .
University Institute of Engineering (UIE) EMPOWER
 Department of Computer Science and Engineering (CSE)

Learning Objectives & Outcomes

Objective:
• To understand the concept of Turing Machines.

Outcome:
• Student will understand the
 Turing machines.
 Hierarchy of Turing machine

University Institute of Engineering (UIE)

 Department of Computer Science and Engineering (CSE)

Turing Machines (TM)

• Generalize the class of CFLs:

Non-Recursively Enumerable Languages

Recursively Enumerable Languages

Recursive Languages

Context-Free Languages

Regular Languages

3
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

• Another Part of the Hierarchy:

Non-Recursively Enumerable Languages

Recursively Enumerable Languages

Recursive Languages

Context-Sensitive Languages

Context-Free Languages - ε

Regular Languages - ε

4
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

• Recursively enumerable languages are also known as

type 0 languages.

• Context-sensitive languages are also known as type 1

languages.

• Context-free languages are also known as type 2

languages.

• Regular languages are also known as type 3 languages.

5
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

• TMs model the computing capability of a general purpose computer,

which informally can be described as:
– Effective procedure
• Finitely describable
• Well defined, discrete, “mechanical” steps
• Always terminates
– Computable function
• A function computable by an effective procedure

• TMs formalize the above notion.

• Church-Turing Thesis: There is an effective procedure for solving a

problem if and only if there is a TM that halts for all inputs and solves the
problem.
– There are many other computing models, but all are equivalent to or subsumed by
TMs. There is no more powerful machine (Technically cannot be proved).

• DFAs and PDAs do not model all effective procedures or computable

functions, but only a subset.
6
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

Deterministic Turing Machine (DTM)

…….. ……..
B B 0 1 1 0 0 B B

Finite
Control
• Two-way, infinite tape, broken into cells, each containing one symbol.
• Two-way, read/write tape head.
• Finite control, i.e., a program, containing the position of the read head,
current symbol being scanned, and the current state.
• An input string is placed on the tape, padded to the left and right infinitely
with blanks, read/write head is positioned at the left end of input string.
• In one move, depending on the current state and the current symbol being
scanned, the TM 1) changes state, 2) prints a symbol over the cell being
scanned, and 3) moves its’ tape head one cell left or right.
• Many modifications possible.

7
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

Formal Definition of a DTM

• A DTM is a seven-tuple:

M = (Q, Σ, Γ, δ, q0, B, F)

Q A finite set of states

Σ A finite input alphabet, which is a subset of Γ– {B}
Γ A finite tape alphabet, which is a strict superset of Σ
B A distinguished blank symbol, which is in Γ
q0 The initial/starting state, q0 is in Q
F A set of final/accepting states, which is a subset of Q
δ A next-move function, which is a mapping (i.e., may be undefined) from
Q x Γ –> Q x Γ x {L,R}

Intuitively, δ(q,s) specifies the next state, symbol to be written, and the direction of
tape head movement by M after reading symbol s while in state q.
8
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

• Example #1: {0n1n | n >= 1}

0 1 X Y B
q0 (q1, X, R) - - (q3, Y, R) -
q1 (q1, 0, R) (q2, Y, L) - (q1, Y, R) -
q2 (q2, 0, L) - (q0, X, R) (q2, Y, L) -
q3 - - - (q3, Y, R) (q4, B, R)
q4 - - - - -

• Sample Computation: (on 0011)

q00011 |— Xq1011
|— X0q111
|— Xq20Y1
|— q2X0Y1
|— Xq00Y1
|— XXq1Y1
|— XXYq11
|— XXq2YY
|— Xq2XYY
|— XXq0YY
|— XXYq3Y
|— XXYYq3
9
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

Making a TM for {0n1n | n >= 1}

Try n=2 or 3 first.

• q0 is on 0, changes the character to X, changes state to q1, moves right
• q1 sees next 0, ignores (both 0’s and X’s) and keeps moving right
• q1 hits a 1, changes it to Y, state to q2, moves left
• q2 sees a Y or 0, ignores, continues left
• when q2 sees X, moves right, returns to q0 for looping step 1 through 5
• when finished, q0 sees Y (no more 0’s), changes to pre-final state q3
• q3 scans over all Y’s to ensure there is no extra 1 at the end
• when q3 sees B, all 0’s matched 1’s, done, changes to final state q4
• blank line for final state q4

Try n=1 next.

Make sure unbalanced 0’s and 1’s, or mixture of 0-1’s,

“crashes” in a state not q4, as it should be

Can you extend the machine to include n=0?

How does the input-tape look like for string epsilon?

University Institute of Engineering (UIE) 10

 Department of Computer Science and Engineering (CSE)

• Same Example #1: {0n1n | n >= 1}

0 1 X Y B
q0 (q1, X, R) - - (q3, Y, R) -
q1 (q1, 0, R) (q2, Y, L) - (q1, Y, R) -
q2 (q2, 0, L) - (q0, X, R) (q2, Y, L) -
q3 - - - (q3, Y, R) (q4, B, R)
q4 - - - - -

– The TM basically matches up 0’s and 1’s

– q1 is the “scan right” state
– q2 is the “scan left” state
– q4 is the final state

• Other Examples:
000111 00
11 001
011

11
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

• Example #2: {w | w is in {0,1}* and w ends with a 0}

0
00
10
10110
Not ε

Q = {q0, q1, q2}

Γ = {0, 1, B}
Σ = {0, 1}
F = {q2}
δ:

0 1 B
q0 (q0, 0, R) (q0, 1, R) (q1, B, L)
q1 (q2, 0, R) - -
q2 - - -

– q0 is the “scan right” state

– q1 is the verify 0 state

12
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

• Exercises: Construct a DTM for each of the following.

– {w | w is in {0,1}* and w ends in 00}

– {w | w is in {0,1}* and w contains at least 2 0’s}
– {w | w is in {0,1}* and w contains at least one 0 and one 1}
– Just about anything else (simple) you can think of

13
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

Formal Definitions for DTMs

• Let M = (Q, Σ, Г, δ, q0, B, F) be a TM.

• Definition: An instantaneous description (ID) is a triple α1qα2, where:

– q, the current state, is in Q

– α1α2, is in Г*, and is the current tape contents up to the rightmost non-blank symbol, or the
symbol to the left of the tape head, whichever is rightmost
– The tape head is currently scanning the first symbol of α2
– At the start of a computation α1= ε
– If α2= ε then a blank is being scanned

• Example: (for TM #1)

q00011 Xq1011 X0q111 Xq20Y1 q2X0Y1

Xq00Y1 XXq1Y1 XXYq11 XXq2YY Xq2XYY

XXq0YY XXYq3Y XXYYq3 XXYYBq4

14
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

• Suppose the following is the current ID of a DTM

x1x2…xi-1qxixi+1…xn

Case 1) δ(q, xi) = (p, y, L)

(a) if i = 1 then qx1x2…xi-1xixi+1…xn |— pByx2…xi-1xixi+1…xn

(b) else x1x2…xi-1qxixi+1…xn |— x1x2…xi-2pxi-1yxi+1…xn

– If any suffix of xi-1yxi+1…xn is blank then it is deleted.

Case 2) δ(q, xi) = (p, y, R)

x1x2…xi-1qxixi+1…xn |— x1x2…xi-1ypxi+1…xn

– If i>n then the ID increases in length by 1 symbol

x1x2…xnq |— x1x2…xnyp

15
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

• Definition: Let M = (Q, Σ, Г, δ, q0, B, F) be a TM, and let w be a string in Σ*. Then w is
accepted by M iff

q0w |—* α1pα2

where p is in F and α1 and α2 are in Г*

• Definition: Let M = (Q, Σ, Г, δ, q0, B, F) be a TM. The language accepted by M,

denoted L(M), is the set

{w | w is in Σ* and w is accepted by M}

• Notes:
– In contrast to FA and PDAs, if a TM simply passes through a final state then the
string is accepted.
– Given the above definition, no final state of an TM need have any exiting
transitions. Henceforth, this is our assumption.
– If x is not in L(M) then M may enter an infinite loop, or halt in a non-final state.
– Some TMs halt on all inputs, while others may not. In either case the language
defined by TM is still well defined.

16
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

• Definition: Let L be a language. Then L is recursively enumerable if there exists a

TM M such that L = L(M).

– If L is r.e. then L = L(M) for some TM M, and

• If x is in L then M halts in a final (accepting) state.
• If x is not in L then M may halt in a non-final (non-accepting) state, or loop forever.

• Definition: Let L be a language. Then L is recursive if there exists a TM M such

that L = L(M) and M halts on all inputs.

– If L is recursive then L = L(M) for some TM M, and

• If x is in L then M halts in a final (accepting) state.
• If x is not in L then M halts a non-final (non-accepting) state.

Notes:

– The set of all recursive languages is a subset of the set of all recursively enumerable
languages

– Terminology is easy to confuse: A TM is not recursive or recursively enumerable, rather a

language is recursive or recursively enumerable.

17
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

• Recall the Hierarchy:

Non-Recursively Enumerable Languages

Recursively Enumerable Languages

Recursive Languages

Context-Sensitive Languages

Context-Free Languages - ε

Regular Languages - ε

18
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

• Observation: Let L be an r.e. language. Then there is an infinite list M0, M1, … of TMs
such that L = L(Mi).

• Question: Let L be a recursive language, and M0, M1, … a list of all TMs such that L
= L(Mi), and choose any i>=0. Does Mi always halt?

• Answer: Maybe, maybe not, but at least one in the list does.

• Question: Let L be a recursive enumerable language, and M0, M1, … a list of all
TMs such that L = L(Mi), and choose any i>=0. Does Mi always halt?

• Answer: Maybe, maybe not. Depending on L, none might halt or some may halt.

– If L is also recursive then L is recursively enumerable, recursive \subset recusrive-enumerable

• Question: Let L be a recursive enumerable language that is not recursive (L is in r.e.

– r), and M0, M1, … a list of all TMs such that L = L(Mi), and choose any i>=0. Does Mi
always halt?

• Answer: No! If it did, then L would not be in r.e. – r, it would be recursive.

19
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

L is Recursively enumerable:
TM exist, M0, M1, …

L is Recursive:
at least one TM halts on L and on ∑* - L, others may or may not

L is Recursively enumerable but not Recursive:

TM exist, but none halts on all x in ∑* - L

L is not R.E:
no TM exist

University Institute of Engineering (UIE) 20

 Department of Computer Science and Engineering (CSE)

• Let M be a TM.

– Question: Is L(M) r.e.?

– Answer: Yes! By definition it is!

– Question: Is L(M) recursive?

– Answer: Don’t know, we don’t have enough information.

– Question: Is L(M) in r.e – r?

– Answer: Don’t know, we don’t have enough information.

21
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

• Let M be a TM that halts on all inputs:

– Question: Is L(M) recursively enumerable?

– Answer: Yes! By definition it is!

– Question: Is L(M) recursive?

– Answer: Yes! By definition it is!

– Question: Is L(M) in r.e – r?

– Answer: No! It can’t be. Since M always halts, L(M) is recursive.

22
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

• Let M be a TM.

– As noted previously, L(M) is recursively enumerable, but may or may not

be recursive.

– Question: Suppose that L(M) is recursive. Does that mean that M always
halts?
– Answer: Not necessarily. However, some TM M’ must exist such that
L(M’) = L(M) and M’ always halts.

– Question: Suppose that L(M) is in r.e. – r. Does M always halt?

– Answer: No! If it did then L(M) would be recursive and therefore not in
r.e. – r.

23
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

• Let M be a TM, and suppose that M loops forever on some string x.

– Question: Is L(M) recursively enumerable?

– Answer: Yes! By definition it is.

– Question: Is L(M) recursive?

– Answer: Don’t know. Although M doesn’t always halt, some other TM M’
may exist such that L(M’) = L(M) and M’ always halts.

– Question: Is L(M) in r.e. – r?

– Answer: Don’t know.
May be another M’ will halt on x, and on all strings! We just do not know!

24
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

Modifications of the Basic TM Model

• Other (Extended) TM Models:

– One-way infinite tapes
– Multiple tapes and tape heads
– Non-Deterministic TMs
– Multi-Dimensional TMs (n-dimensional tape)
– Multi-Heads
– Multiple tracks

All of these extensions are equivalent to the basic TM model

25
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

REFERENCES :
•Hopcroft J.E. and Ullman J.D., “Introduction to Automata Theory
Languages and Computation”, Narosa Publications.

University Institute of Engineering (UIE) 26

 Department of Computer Science and Engineering (CSE)

UNIVERSITY INSTITUTE OF
ENGINEERING
COMPUTER SCIENCE
ENGINEERING
Bachelor of Engineering
Theory of Computation (CST-353)

Topic: Unsolvable Problem, PCP, MPCP

DISCOVER . LEARN .
University Institute of Engineering (UIE) EMPOWER
 Department of Computer Science and Engineering (CSE)

Learning Objectives & Outcomes

Objective:
• To understand the concept of unsolvable problem.

Outcome:
• Student will understand the
 Decidable and undecidable problem.
 Halting problem
 PCP
 MPCP

University Institute of Engineering (UIE)

 Department of Computer Science and Engineering (CSE)

Introduction to Undecidability

• In the theory of computation, we often come across such

problems that are answered either 'yes' or 'no'. The class
of problems which can be answered as 'yes' are called
solvable or decidable. Otherwise, the class of problems is
said to be unsolvable or undecidable.

University Institute of Engineering (UIE)

 Department of Computer Science and Engineering (CSE)

Theorem: Lu is RE but not recursive.

University Institute of Engineering (UIE)

 Department of Computer Science and Engineering (CSE)

Undecidable Languages

University Institute of Engineering (UIE)

 Department of Computer Science and Engineering (CSE)

The Halting Problem - Background

• Definition: A decision problem is a problem having a yes/no answer (that one

presumably wants to solve with a computer). Typically, there is a list of parameters on
which the problem is based.
– Given a list of numbers, is that list sorted?
– Given a number x, is x even?
– Given a C program, does that C program contain any syntax errors?
– Given a TM (or C program), does that TM contain an infinite loop?

From a practical perspective, many decision problems do not seem all that interesting.
However, from a theoretical perspective they are for the following two reasons:
– Decision problems are more convenient/easier to work with when proving complexity results.
– Non-decision counter-parts are typically at least as difficult to solve.

• Notes:
– The following terms and phrases are analogous:

Algorithm - A halting TM program

Decision Problem - A language
(un)Decidable - (non)Recursive

6
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

Statement of the Halting Problem

• Practical Form: (P1)
Input: Program P and input I.
Question: Does P terminate on input I?

• Theoretical Form: (P2)

Input: Turing machine M with input alphabet Σ and string w in Σ*.
Question: Does M halt on w?

• A Related Problem We Will Consider First: (P3)

Input: Turing machine M with input alphabet Σ and one final state, and string w in
Σ*.
Question: Is w in L(M)?

• Analogy:
Input: DFA M with input alphabet Σ and string w in Σ*.
Question: Is w in L(M)?
Is this problem (regular language) decidable? Yes! DFA always accepts or rejects.

7
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

• Over-All Approach:

– We will show that a language Ld is not recursively enumerable

– From this it will follow that is not recursive
– Using this we will show that a language Lu is not recursive
– From this it will follow that the halting problem is undecidable.
Ld

• As We Will See:
– P3 will correspond to the language Lu
– Proving P3 (un)decidable is equivalent to proving Lu (non)recursive

8
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

Converting the Problem to a Language

• Let M = (Q, Σ, Γ, δ, q1, B, {qn}) be a TM, where

Q = {q1, q2, … , qn}

Σ = {x1, x2} = {0, 1}
Γ = {x1, x2, x3} = {0, 1, B}

• Encode:

δ(qi, xj) = (qk , xl, dm) where qi and qk are in Q

xj and xl are in Σ,
and dm is in {L, R} = {d1, d2}
as:
0i10j10k10l10m

• The TM M can then be encoded as:

111code111code211code311 … 11coder111

where each codei is one transitions’ encoding. Let this encoding of M be denoted by <M>.

9
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

• Less Formally:

– Every state, tape symbol, and movement symbol is encoded as a sequence of 0’s:
q1, 0
q2, 00
q3 000
:

0 0
1 00
B 000

L 0
R 00

– Note that 1’s are not used to represent the above, since 1 is used as a special separator symbol.

– Example:

δ(q2, 1) = (q3 , 0, R)

Is encoded as:

00100100010100
10
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

0 1 B
q1 (q1, 0, R) (q1, 1, R) (q2, B, L)
q2 (q3, 0, R) - -
q3 - - -

1110101010100110100101001001101000100100010110010100010100111

01100001110001

111111

11
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)
• Definition:

Lt = {x | x is in {0, 1}* and x encodes a TM}

– Question: Is Lt recursive?
– Answer: Yes. [Check for format, i.e. order and number of 0’s and 1’s, syntax
checking]

– Question: Is Lt decidable:
– Answer: Yes (same question).

12
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

The Universal Language

• Define the language Lu as follows:

Lu = {x | x is in {0, 1}* and x = <M,w> where M is a TM encoding and w is in L(M)}

• Let x be in {0, 1}*. Then either:

1. x doesn’t have a TM prefix, in which case x is not in Lu

1. x has a TM prefix, i.e., x = <M,w> and either:

a) w is not in L(M), in which case x is not in Lu

a) w is in L(M), in which case x is in Lu

13
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

• Recall:

0 1 B
q1 (q1, 0, R) (q1, 1, R) (q2, B, L)
q2 (q3, 0, R) - -
q3 - - -

• Which of the following are in Lu?

1110101010100110100101001001101000100100010110010100010100111

111010101010011010010100100110100010010001011001010001010011101110

11101010101001101001010010011010001001000101100101000101001110011011
1

01100001110001

111111

14
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

• Compare P3 and Lu:

(P3):
Input: Turing machine M with input alphabet Σ and one final state, and string w in
Σ*.
Question: Is w in L(M)?

Lu = {x | x is in {0, 1}* and x = <M,w> where M is a TM encoding and w is in L(M)}

• Universal TM (UTM) works on Lu

• Notes:
– Lu is P3 expressed as a language
– Asking if Lu is recursive is the same as asking if P3 is decidable.
– We will show that Lu is not recursive, and from this it will follow that P3 is un-
decidable.
– From this we can further show that the halting problem is un-decidable.
– Note that Lu is recursive if M is a DFA.

15
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

• Define another language Ld as follows:

Ld = {x | x is in {0, 1}* and (a) either x is not a TM,

(b) or x is a TM, call it M, and x is not in L(M)} (1)

• Let x be in {0, 1}*. Then either:

1. x is not a TM, in which case x is in Ld

1. x is a TM, call it M, and either:

a) x is not in L(M), in which case x is in Ld

a) x is in L(M), in which case x is not in Ld

16
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

• Recall:

0 1 B
q1 (q1, 0, R) (q1, 1, R) (q2, B, L)
q2 (q3, 0, R) - -
q3 - - -

• Which of the following are in Ld?

11101010101001101001010010011010001000100010110010100010100111

01100001110001

111111

17
University Institute of Engineering (UIE)
 Department of Computer Science and Engineering (CSE)

Post Correspondence Problem

• In this section, we will discuss the undecidability of string and not of
Turing machines. The undecidability of the string is determined with
the help of Post's Correspondence Problem (PCP).
• The Post Correspondence Problem (PCP), introduced by Emil Post in
1946, is an undecidable decision problem. The PCP problem over an
alphabet ∑ is stated as follows –
•
• "The Post's correspondence problem consists of two lists of string
that are of equal length over the input. The two lists are :
• A = w1, w2, w3, .... , wn
• B = x1, x2, x3, .... xn
• then there exists a non empty set of integers i1, i2, i3, .... , in such that,
w1, w2, w3, .... wn = x1, x2, x3, .... xn"

University Institute of Engineering (UIE)

 Department of Computer Science and Engineering (CSE)

Example 1: PCP

• Find whether the lists

• M = (abb, aa, aaa) and N = (bba, aaa, aa)
• have a Post Correspondence Solution?

x1 x2 x3

M abb aa aaa

N bba aaa aa

University Institute of Engineering (UIE)

 Department of Computer Science and Engineering (CSE)

Example 1: PCP (Contd…)

• Here,
• x2x1x3 = ‘aaabbaaa’
• and y2y1y3 = ‘aaabbaaa’
• We can see that
• x2x1x3 = y2y1y3
• Hence, the solution is i = 2, j = 1, and k = 3.

University Institute of Engineering (UIE)

 Department of Computer Science and Engineering (CSE)

Modified Post Correspondence Problem

• In MPCP, there is the additional requirement on a

solution that the first pair on the A and B lists must be
the first pair in the solution.
• More formally, an instance of MPCP is two lists A=w1,w2,
w3, …….wk and B= x1, x2 ,x3, ……xk for some integer k.
• And a solution is a list of 0 or more integers i1, i2,…., im
Such that w1wi1 wi2 ……. wim = x1 xi1 xi2 ……. xim .

University Institute of Engineering (UIE)

 Department of Computer Science and Engineering (CSE)

Why MPCP ?

• We use MPCP as a medium for proving PCP is

Undecidable.
• • First we reduce Lu (ATM ) to MPCP instance.
• • We Reduce MPCP instance to PCP instance.
• • As from reductions from Lu to PCP as MPCP in middle It
become easy to prove that PCP is Undecidable.

University Institute of Engineering (UIE)

 Department of Computer Science and Engineering (CSE)

Reducing MPCP To PCP ! Rules for conversion of MPCP

to PCP

University Institute of Engineering (UIE)

 Department of Computer Science and Engineering (CSE)

REFERENCES :
•Hopcroft J.E. and Ullman J.D., “Introduction to Automata Theory
Languages and Computation”, Narosa Publications.
•https://www.javatpoint.com/introduction-to-undecidability
•https://www.tutorialspoint.com/automata_theory/post_correspondence_
problem.htm
•https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.721.2199&r
ep=rep1&type=pdf

University Institute of Engineering (UIE) 24