Theory of Computation

• It is a branch of computer science that deals with how efficiently a problem can be solved
Automata and Complexity Theory
on a model of computation, using an algorithm.
• Theory of computation is divided into 3 fields:
1. Automata Theory and Formal Languages
Chapter 1:Introduction 2. Computability Theory
3. Complexity Theory
• The goal is to ascertain the power and limits of computation.
• In order to study these aspects, it is necessary to define precisely what constitutes a model of
computation as well as what constitutes a computational problem.
• This is the purpose of automata theory.
• The computational models are automata, while the computational problems are formulated as
formal languages. 2

Cont’d...
Alphabets and Strings
1. Automata Theory and Formal Languages 1. Symbol
• It deals with the definition and properties of various mathematical model of computation. which can be any alphabet, letter, or picture.
• Examples: • Example
i. Finite Automata(FA) • a, b, c, …, z
ii. Context Free Grammar (CFG) • 0,1,…, 9
iii. Turing Machine (TM)
• +,*, #, … special characters
2. Computability Theory
2. Alphabet
• It deals with what can/cannot be computed by the model.
• An alphabet is a finite, non-empty set of symbols.
3. Complexity Theory
• It groups the computable problems based on their hardness.
• It is denoted by ∑.

Main purpose of Theory of Computation • Examples: ∑ = {0,1} set of binary alphabets. 4

• Develop formal mathematical model of computation that reflect real world computers.
3

• ymbol(often also called character) is the smallest building block,

 Cont'd…. Cont'd….
• Examples: ∑ = {a, b, c, …, z} set of all lowercase letters. 4. Empty String

∑ = {A, B, C, …Z} is an alphabet of uppercase letters. • Empty string is a string with zero occurrences of symbol(no symbol)..

∑ = {+, #, &,$, …} set of all special characters • It is denoted by ' ε ' → no symbol { }

3. String or Word 5. Length of String

• It is a finite collection of symbols from the alphabet. • It is the number of symbols in the string.

• The string is denoted by w. • It is denoted by |w|.

• Example: w = 01110 is string from binary alphabet ∑ = {0,1} • Example 1: w = 010110101 from binary alphabet ∑ = {0,1}

w = aabbaacaa is string from alphabet ∑ = {a, b, c} Length of the string |w| =9.
Example 2: w = 010
Symbol Alphabet String/Word Length of Sting |w| = 3
5 6

Cont'd…. Cont'd….
6. Concatenation of String : joining of 2 or more strings. • Example: ∑ = {0,1} has 2 symbols.

• Let x = a1, a2, a3, …, an (i) ∑1 = {0,1} therefore 21 = 2 where k = 1

y= b1,b2,b3, …, bn (ii) ∑2 = {00,01, 10,11} therefore 22 = 4 where k = 2

• Concatenation of string xy = a1, a2, a3, …, anb1,b2,b3, …, bn (ii) ∑3 = {000, 001, 010, 011, 100, 101, 110, 111 } therefore 23 = 8

• Example: S = ababa and T = cdcddc where k = 3

Concatenation of string ST = ababacdcddc • The set of strings over an alphabet ∑ is usually denoted by ∑* is kleen

7. Power of an Alphabet closure.

• If ' ∑ ' is an alphabet, we can express set of all string of certain length from • For instance ∑* = {0,1}*

that alphabet by using exponential notation. = {ε,0,1,00,01,10,11,…}

• It is denoted by ∑k is the set of strings of length k. • Therefore, kleen closure ∑* = ∑0 U ∑1 U ∑2 U ∑3 …

7 8
 Cont'd…. Example: 1 Cont'd….
• The set of strings over an alphabet ∑ excluding ε is denoted by ∑+ is called
• L1 = {Set of string of length 2} = {aa, bb, ba, bb} Finite Language
kleen plus.
• L2 = {Set of all strings starts with 'a'} = {a, aa, aaa, abb, abbb, ababb} Infinite
• For instance ∑+ = {0,1}+
Language
= {0,1,00,01,10,11,…}
Grammar:
• Therefore, kleen plus ∑+ = ∑* - {ε } or
• A grammar is finite set of rules defining a language.
∑+ = ∑* - {ε }
8. Language:
• A language is a collection of appropriate string.
• A language which is formed over Σ can be Finite or Infinite.
9 10

Automata Finite Automata(FA)

• Theory of automata is a theoretical branch of computer science and mathematical. • Finite automata are used to recognize patterns.
• It is the study of abstract machines and the computation problems that can be solved • It takes the string of symbol as input and changes its state accordingly.
using these machines. • When the desired symbol is found, then the transition occurs.
• The abstract machine is called the automata. • At the time of transition, the automata can either move to the next state or stay in
• The main motivation behind developing the automata theory was to develop
the same state.
methods to describe and analyze the dynamic behavior of discrete systems.
• Finite automata have two states:
• This automaton consists of states and transitions.
• Accept state or
• The State is represented by circles, and the Transitions is represented by arrows.
• Reject state.
• Automata is the kind of machine which takes some string as input and this input
• When the input string is processed successfully, and the automata reached
goes through a finite number of states and may enter in the final state.
11
its final state, then it will accept, otherwise it will reject the input. 12
 Cont’d …
Formal Definition of FA

• Finite automata can be represented by: • A finite automaton is a collection of 5-tuples (Q, ∑, δ, q0, F), where:

• Graphical (Transition diagrams) • Q is finite set of states

• Tabular (transition tables) • ∑ is finite set of the input alphabets

• Mathematical (Transition function or Mapping function) • q0 ϵ Q is start state also called initial state
• F is accept state or final state
• δ: Q x ∑ → Q is transition function

13 14

Transition Diagrams Transition Table

• Transition diagram is a directed graph associated with vertices of the graph • Transition Table is basically a tabular representation, of the transition function that
corresponds to the state of finite automata. takes two arguments( a state and a symbol ) and returns a value (next state).
1
1 • Rows corresponds to states,
0
• Example: • Columns corresponds to input symbols
1 0
q0 q1 q2 • Entries corresponds to next state
Start or Final or • The start state is marked with an arrow (→)
Initial State Accept State
• Accept state are marked with a star(*)
• {0, 1} are inputs
• q1 intermediate state
• qo initial state
• q1 intermediate state • q2 is final state
• q2 is final state 15 16
 Cont’d …
Transition Function
• Example:
• The mapping function or transition function denoted by δ.
0, 1
0 ➢ Two parameters are passed to this transition function
0 1 1. current state
q0 1 q1
→qo q0 q1 2. input symbols
Transition diagram
* q1 q1 q1 • Transition function always returns a state which can be called as next state.
δ(current state, current input symbol) = next state
• qo initial state Transition table Qx∑→Q
• q2 is final state
• {0, 1} are inputs • Example: δ(q0, a) = q1
δ(q0, 1) = q1
17 18

Types of Automata Deterministic Finite Automata (DFA)

• Finite automata is called deterministic finite automata if the machine read an
Finite Automata
input string one symbol at a time.
• Deterministic refers to the uniqueness of the computation.
Finite Automata without output Finite Automata with output
• In DFA, there is only one path for specific input symbol from current state
to the next state.
• DFA does not accept the null move.
Deterministic Non-deterministic Non-deterministic Mealy
Moore
Finite Automata Finite Automata Finite Automata Machine • DFA cannot change the state without any input character.
Machine
without output without output with ε - Moves
(DFA) (NFA) • DFA can contains multiple final state.
• DFA is used in lexical analysis in compiler.
19 20
 Cont’d …
Formal Definition of DFA
a, b
• A DFA is a collection of 5 tuples (same as FA)

q0 b q1 ➢ Q is finite set of states

➢ ∑ is finite set of input symbols
a
➢ q0 is initial state
q2 qo is initial state
➢ F is final state
q1 is final state
➢ δ is transition function
{a, b} input symbols
• Transition function can be defined as δ : Q x ∑ → Q

21 22

Cont’d … Cont’d …

• The state is represented by vertices. • DFA to accept 'a'

• The arc labeled with an input character show the transition q0 a q1
• The initial state is marked with an arrow.
• The final state is denoted by a double circle. a
Acceptance of a Language
• A language acceptance is defined by “if a string w is accepted by a machine m” • DFA to accept zero or more 'a' q0
• If it is reaching the final state F by taking the string w. • L = { ∑, a, aa, aaa, aaaa, …. }
• Not accepted if not reaching the final state.
• Example: L={Ø} → q0 no string • Note : states depends on the length of the string.
L={∑} → q0 .
23 24
 Cont’d … Cont’d …

• Example 1: Let DFA be Q = {q0, q1,q2}, ∑ = {0, 1}, initial state ={q0}and final • Example 2: Design DFA with given ∑ = {0,1} that accepts those string which starts
state ={q2. with 1 and ends with 0.
• So, Construct DFA for the below table. • Solution: minimum length of string is 2.
Transition table L = { 10, 100, 1010, 11001100, …. }
Transition Diagram
Present State 0 1 Number of states = |minimum length| + 1
→q0 q0 q1
q1 q2 q0 DFA Diagram: 0
Solution: 1
*q2 q1 q2
q0 1 q1 0 q2
1
25 26

Cont’d …

• Example 3: Construct DFA that accepting all strings over ∑ = {a,b} ending with Non Deterministic Finite Automata (DFA)

ab. • The finite automata are called NFA when there exist many paths for specific input
• Solution: minimum length of string is 2. from the current state to the next state.
number of states = 2+1 =3 • It is easy to construct an NFA than DFA for a given regular language.
L = {ab, aab, bab, bbab…. } • Every NFA is not DFA, but each NFA can be translated into DFA.

b • NFA is defined in the same way as DFA but with the following two exceptions:
b a
DFA Diagram: 1. it contains multiple next states, and
2. it contains ε transition.
q0 a q1 b q2
a

b 27 28
 Cont’d … Cont’d …
Formal definition of NFA
• In the following image,
• NFA also has five tuples same as DFA, but with different transition function, as
• we can see that from state q0 for input a, there are two next states q1 and q2
shown follows:
• similarly, from q0 for input b, the next states are q0 and q1.
δ: Q x ∑ →2Q
• Thus it is not fixed or determined that with a particular input where to go next.
• where,
• Hence this FA is called non-deterministic finite automata.
• Q finite set of States
• ∑ finite set of input symbol
• q0 initial state
• F final state
• δ transition function

29 30

Cont’d … Cont’d …
Graphical Representation of an NFA • Transition Table for the above NFA diagram
• An NFA can be represented by digraphs called state diagram. In which: Present State Next state for Input 0 Next State of Input 1
❖ The state is represented by vertices. →q0 q0, q1 q1
❖ The arc labeled with an input character show the transitions. q1 q2 q0
❖ The initial state is marked with an arrow. *q2 q2 q1, q2
❖ The final state is denoted by the double circle. • In the above diagram, we can see that when the current state is q0, on input 0, the next state

• Example 1: Construct transition diagram of NFA from the below given information. will be q0 or q1, and on 1 input the next state will be q1.
• When the current state is q1, on input 0 the next state will be q2 and on 1 input, the next
Q = {q0, q1, q2}
∑ = {0, 1} state will be q0.
q0 = {q0} • When the current state is q2, on 0 input the next state is q2, and on 1 input the next state
F = {q2} will be q1 or q2.
31 32
 Cont’d … Cont’d …
• Example 2: NFA with ∑ = {0, 1} accepts all strings starts with 01. • Example 3: NFA with ∑ = {0, 1} and accept all string of length at least 2.
• Solution: • Solution:

• Transition Table:
• Transition Table:
Present State Next state for Input 0 Next State of Input 1
Present State Next state for Input 0 Next State of Input 1
→q0 q1 q1
→q0 q1 ε
q1 q2 q2
q1 ε q2
*q2 ε ε
*q2 q2 q2
33 34

Cont’d … Cont’d …
• The transition diagram can be drawn by using the mapping function as:
• Example 4: Design a NFA for the transition table as given below:

Present State 0 1
→q0 q0, q1 q0, q2
q1 q3 ε
q2 q2, q3 q3
*q3 q3 q3

35 36
 Cont’d … Cont’d …

• Example 5: Design an NFA with ∑ = {0, 1} accepts all string ending with 01. • Example 6: Construct NFA for set of strings of all that starts with 0 over {0, 1}
• Solution: Solution:

0,1

q0 0 q1

37 38

NFA with ε Difference between DFA and NFA

• ε is empty symbol( no character to accept) DFA NFA
• Regular NFA with 5 tuples {Q, ∑, q0, F, δ} The transition from a state is to a single The transition from a state can be to
particular next state for each i/p symbol. multiple next state for each i/p symbol.
where δ : Q x ∑ → 2Q Hence it is called deterministic. Hence, it is called non-deterministic.
• ε -NFA with 5 tuples {Q, ∑, q0, F, δ} Empty string transitions are not seen in NFA permits empty string transitions.
DFA.
where δ : Q x ∑ U ε → 2Q 1 0 DFA is a subset of NFA. Need to convert NFA to DFA in the
design of compiler.
• Example: 0 ε δ: Q x ∑ →Q δ: Q x ∑ →2Q
A B C
Example: δ(q0, a) = q1 Example: δ(q0,a) = {q1, q2}

B on seeing nothing, it can go to ‘C’ Requires more space Requires less space
A string is accepted by a DFA, if it A string is accepted by a NFA, if at least
Note: Every state on ε goes to itself
transits to a final state. one of all possible transitions ends in a
39
final state. 40
 Cont’d …
Conversion of NFA to DFA Step 1: Construct NFA transition table a b

DFA: δ: Q x ∑ →Q a a b q0 q0 [q0,q1]
NFA DFA:
NFA: δ: Q x ∑ →2Q q0 q0 q0,q1 [q0,q1] [q0] [q0q1q2]

b b q1 - q2 [q0q1q2] [q0] q0q1q2]

Example 1: q0 q1 q2
q2 - - q2 - -
• [q0,q1] new single state
Step 1:construct NFA transition table
a b a b
q0 q0 q0,q1
q1 - q2 q0 b q0q1 b q0q1q2
q2 - -
a
41
a 42

Cont’d … Cont’d …
Example 2: Convert the given NFA to DFA Step 2: Construct DFA transition table and create new states
NFA DFA
a, b b
δ a b δ a b
a, b →q0 {q0,q1} q0 →q0 [q0,q1] q0
q0 a q1 a q2 q3
q1 q2 q1 [q0q1] [q0q1q2] q0q1
Step 1:construct NFA transition table *q2 q3 q3 *[q0q1q2] [q0q1q2q3] [q0q1q3]
b
*q3 - q2 *[q0q1q2q3] [q0q1q2q3] [q0q1q3q2]
δ a b
*[q0q1q3] [q0q1q2] [q0q1q2]
→q0 {q0,q1} q0 b b
q1 q2 q1 a a a, b a, b
*q2 q3 q3 q0 q0,q1 q0,q1,q2 q0,q1,q2,q3
*q3 - q2 b a, b
43
q0,q1,q3 44
 Cont’d …
Problem:
Equivalence of Two Finite Automata
Convert the following Non-Deterministic Finite Automata (NFA) to Deterministic
Finite Automata (DFA): • For any pair of states(qi, qj) the transition for input a ε ∑ is defined by
{qa, qb} where δ{qi,a}=qa and δ{qj,a}=qb.
• The two automata are not equivalent if for a pair {qa, qb} one is
intermediate state and the other is final state.
• If initial state is final state of one automata, then second automata
also initial state must be final state for them to be equivalent.

45 46

Cont’d … Cont’d …
Example1: find out whether the following two finite automata are equivalent or not. State/ Input c d
c
c d {q1, q4} {q1, q4} {q2, q5}
d FS, FS IS, IS
A q2 q3
q1
{q2, q5} {q3, q6} {q1, q4}
c IS, IS FS, FS
d
{q3, q6} {q2, q7} {q3, q6}
c IS, IS IS, IS
d
q7 • Pair of states both are {q2, q7} {q3, q6} {q1, q4}
q4
B IS, IS FS, FS
c c final state or initial state
d
d
are equivalent.
q5 c q6 d • Based on the above table , the given finite automata A and B are equivalent.
47 48
 Cont’d … Cont’d …
Example2: find out whether the following two finite automata are equivalent or not.
c State/ Input c d
c d
d {q1, q4} {q1, q4} {q2, q5}
A q1 q2 q3
FS, FS IS, IS
c {q2, q5} {q3, q7} {q1, q6}
d
IS, IS FS, IS
c
c
q7 • Pair of states both are • Based on the above table , the given finite automata A and B are not equivalent.
q4
B
c final state or initial state
c
d c
or intermediate state are
q5 d q6 d equivalent.
49 50

Cont’d …
Minimization of DFA
Step5: Repeat step 3 until we find no similar rows available in T1
• The minimization of DFA means reducing the number of states from a given FA.
Step6: Repeat step 3 and step 4 for table T2 also.
Step1: Remove all the states that are unreachable from initial state via any set of
Step7: Now combine the reduced T1 and T2 tables.
transition of DFA.
i.e.. the final transition table of minimized DFA.
Step2: Draw the transition table for all pair of states.
• Example 1: Minimization of DFA
Step3: Now split the transition table into two tables T1 and T2.
0 q1 0
T1 contains all final states q0
q2 1
T2 contains non-final states 0 1 q4
1 0, 1
• Step4: Find similar rows from T1, such that:
q3
δ(q, a) =p • Find the two states which have same value of a and b and
0, 1
δ(r, a) =p remove one of them.
q5 0, 1
51 52
 Cont’d … Cont’d …
Step1: Remove q2 and q4 in FA because they are unreachable states. Step3: (i) tables which starts from non final states.
States 0 1
0 q1 →q0 q1 q3
q0
T1
0 q1 q0 q3
1
1
0, 1 q5 (ii) tables which starts from final states.
q3 0, 1 States 0 1
Step2: Draw transition table for the rest of the states. *q3 q5 q5
T2
States 0 1 *q5 q5 q5
→q0 q1 q3 T1 • Step4: (i) set 1 has no similar rows, they will be same.
q1 q0 q3 (ii) set 2 has similar rows, so skip q5 and then replace q5 by q3.
*q3 q5 q5 States 0 1
*q5 q5 q5 T2 *q3 q3 q3
53 54

Cont’d …
Step7: Combine set 1 and set 2 Application of Finite Automata
• Finite Automata (FA) used:
States 0 1
→q0 q1 q3 1. For the designing of lexical analysis of a compiler.
q1 q0 q3 2. For recognizing the pattern using regular expressions.
*q3 q3 q3
3. For the designing of the combination and sequential circuits

0 q1 using Mealy and Moore Machines.

• The minimized DFA q0 4. Used in text editors.
0
1 5. For the implementation of spell checkers.
1
q3 0, 1

55 56
 Activity1 Activity2
1. Draw a DFA for the language accepting strings ending with '01' over 1. Design an NFA in which all the string contain a substring 1110.
input alphabets ∑ = {0, 1}. 2. Design an NFA with ∑ = {0, 1} accepts all string in which the third
2. Draw a DFA for the language accepting strings ending with 'abba' over symbol from the right end is always 0.
input alphabets ∑ = {a, b. 3. Design an NFA with ∑ = {0, 1} in which double '1' is followed by double
3. Draw a DFA for the language accepting strings ending with '0011' over '0’.
input alphabets ∑ = {0, 1}. 4. Design an NFA with ∑ = {a, b,c} in which a is followed by b and
4. Draw a DFA for the language accepting strings starting with ‘0011’ over followed by c.
input alphabets ∑ = {0, 1}.

57 58

Activity3 Activity4
1. Find out whether the following two finite automata are equivalent or not. 1. Minimize the given DFA:

A.

2. Minimize the given DFA:

B.

59 60
 Regular Expressions and Regular Languages

1. Regular Expressions
• Regular Expressions are used to denote regular languages.
• The language accepted by finite automata can be easily described by
Chapter 2: Regular Expression
simple expressions called regular expressions.
and Regular Language
• It is most effective way to represent any language.
• The languages accepted by some regular expressions are referred to as
regular languages.
• A regular expressions can also be described as a sequence of pattern that

12/8/2022 Fikru Tafesse(MSc. in Computer Science) 61

defines a string. 62

Cont’d …
• An expression is regular if: Regular Grammar

o ɸ is a regular expression for regular language ɸ. • A grammar is regular if it has rules of form A → a or A →aB or A → ɛ where ɛ is a special

o ɛ is a regular expression for regular language {ɛ}. symbol called NULL.

• Grammar is set of rules used to describe the string of the language.
o If a Σ (Σ represents the input alphabet), a is regular expression with language
• It is type 3 grammar recognized using Finite Automata.
{a}.
• Formal Definition
o If a and b are regular expression, a + b is also a regular expression with
• Regular Grammar G is defined as a collection of four tuples.
language {a,b}.
G = {V, T, P, S }
o If a and b are regular expression, ab (concatenation of a and b) is also regular.
• V is finite set of symbols called non terminals or variables (Capital letters) which are used
o If a is regular expression, a* (0 or more times a) is also regular.
to define rules.
• T is finite set symbols called terminals(small letters, numbers, operators, etc.)
• S where S ϵ V, it is the start symbol of the grammar.
63
64
 Cont’d … Cont’d …
• P is a set of production rules and it is the form of A → aB, A → a • From Example 2 above: S → aS|bS|a
S
• Where A and B are nonterminal sym
A→a
bols and a is a terminal symbols. S S S
• Example1: aS
a aS bS
• G={V,T,P,S) with V=(S,B), T={0,1} a
{a} bS
• P is given by S → 0S | 1B aS a
B→ε aS
S → 0S S → 0S a {ba}
• Derivation of string 001. • L= (a+b)*a a
→ 00S S → 1B {aaa}
• Strings of the language are: {abaa}
→ 001B B→ε
{a, aaa, aba, aaba} = ending with a.
→ 001ε
→ 001
65 66

Cont’d … Cont’d …
• Problems: Identify languages generated by grammars
• Example 1: S → aS|bS|ε
3. Regular Languages :
• Minimum string is ε.
• A language is regular if it can be expressed in terms of regular expression.
• The grammar can generate any combination of a and b with including empty string.
Note :
• Strings generated by the above grammar are:
• Two regular expressions are equivalent if languages generated by them are
• Strings are: {ε, a, b, aa, bb, ab, ba, abb, aab, aba, abaa, …}
• Language L = (a+b)*
same.

• Example 2: S → aS|bS|a|b • For example, (a+b*)* and (a+b)* generate same language.

• Minimum string is a and b without including ε. • Every string which is generated by (a+b*)* is also generated by (a+b)* and
• String generated by the above grammar are: vice versa.
• {a, b, aa, bb, ab, ba, abb, aab, …} Language L =(a+b)+
67 68
 Cont’d … Cont’d …
• For instance:
• Kleen Closure: if L is regular languages then its kleen closure L* also is
• Regular expression R.E x means zero or more occurrence of x.
*
regular language.
• It can generate {ε, x, xx, xxx, xxxx, …..}
L* = zero or more occurrences of language L.
• Regular expression R.E x+ means one or more occurrence of x.
• Kleen Plus: if L is regular languages then its kleen plus L+ also is regular
• It can generate {x, xx, xxx, xxxx, …..}
language.
Operations on regular
L+ = one or more occurrences of language L.
1. union: if L and M are two regular languages their union L U M is also regular
L U M = {S | S is in L or S is in M}
2. Intersection: if L and M are two regular languages their intersection L ո M is also
regular
L ո M = {ST | S is in L or T is in M} 69 70

Cont’d … Cont’d …
❖ Example 1: Write regular expression for the language accepting all strings ❖ Example 3: Write regular expression for the language accepting all strings

containing any number of a’s and b’s. containing any number of a’s and b’s except empty string.
Solution: R.E = (a+b)* Solution: R.E = (a+b)+
• This will give the set as L= {ε, a, aa, ab, b, aba, ba, bb, …} any combination of • This will give the set as L= {a, aa, ab, b, aba, ba, bb, …} any combination of a
a and b. and b.
• The (a+b)* shows any combination with a and b even ε. • The (a+b)+ shows any combination with a and b even ε.
❖ Example 2: Write regular expression for the language accepting all strings which ❖ Example 4: Write regular expression for the language starting with a, but not

starting with 1 and ending with 0 over ∑ = {0,1}. having consecutive b’s.
Solution: the starting symbol is 1 and ending symbol is 0. Solution L = {a, ab, aab, aba, aaa, abab, ….. }
R.E = 1(0+1) * 0 R.E = (a+ab)*
L= {10, 100, 1110, 110, 1010, …..} 71 72
 Cont’d …
❖ Example 5: Write R.E for the language over ∑ = {0}having Even length of Activity 1

the string 1. Write RE for the language accepting strings ending with '01' over input
Solution: L={ε, 00, 0000, 000000, ….} alphabets ∑ = {0, 1}.
R.E = (00)*
2. Write RE for the language accepting strings ending with 'abba' over input
• This will give the set as L={ε, 00, 0000, 000000, ….} even number of
alphabets ∑ = {a, b}.
zeros.
3. Write RE for the language accepting strings ending with '0011' over input
❖ Example 6: Write R.E for the language over ∑ = {0,1}such that all the
alphabets ∑ = {0, 1}.
strings do not contain the substring 01.
4. Write RE for the language accepting strings ending with ‘0011’ over input
Solution: L={ε, 0, 1,, 00, 10,100, ….}
alphabets ∑ = {0, 1}.
R.E = (1*0*) Note : RE stands for regular Expression
73 12/8/2022 74

Activity 1 Regular Expression(RE) to Finite Automata(FA)

• Rules for converting RE to FA
R1
1. Write RE in which all the string contain a substring 1110. 1. Union: R1 + R2
2. Write RE with ∑ = {0, 1} accepts all string in which the third symbol from the

right end is always 0. R2

3. Write RE with ∑ = {0, 1} in which double '1' is followed by double '0’. 2. Concatenation: R1.R2
R1 R2
4. Write RE with ∑ = {a, b,c} in which a is followed by b and followed by c.

3. Closure: R1* R1

12/8/2022 Note : RE stands for regular Expression 75 76

 ab
• Example1: ab + ac • Example 2: a(ab)*b
ab
a b
q0 q1 qf
ac

a b
a(b + c) This NFA b/c two next q2
a states for input a
c a b
b
a b
a q0 q1 qf
Now this is DFA

c
77 78

• Example 3: (aba)* Method 2

aba • To convert RE to FA, we use a method called subset method.

qf Step1: Design a transition diagram for a given RE using NFA with ε- moves.
Step2: Convert NFA with ε- moves to NFA without ε- moves.
Step3: Convert the obtained to equivalent DFA.
a q1

qf Example1 : Design FA from the given R.E 10+(0+11)0*

b
a
q2 Step1:

79 80
 Cont’d … Cont’d …

Step4:
Step2:

NFA-ε

Step5:
Step3:

81 82

Cont’d … Cont’d …

• Now we have got NFA without ε. • Now finally construct DFA transition diagram
• Now convert NFA to equivalent DFA
q0 q1 q2
• First write transition table for NFA DFA 1
q1q2
NFA 0
δ 0 1 0
0 1 1
δ 0 1 →[q0] [q3] [q1,q2] qf
→q0 q3 q1,q2 [q1] qf - q3
1
q1 qf - [q2] - q3
0
q2 - q3 [q3] q3 qf
q3 q3 qf [q1,q2] qf q3 • From the above diagram we observe that state q1 and q2 are unreachable, so we
*qf - - *qf - - ignore those two states.
83 84
 Cont’d …
Activity 2
• After q1 and q2 are eliminated, DFA will be look like below:
1. Construct the FA for Regular Expression for RE= 0(10)*
q0
2. Convert the following regular expression RE = (a+b)+
1
q1q2 3. Convert the following regular expression for identifier RE = letter(letter +
0 digit)*.
0 1
qf
q3 1 4. Find the FA for regular expression RE= (a+b)*abb

85 12/8/2022 86

Cont’d …
Finite Automata(FA) to Regular Expression(RE)
3. There is only one final state.
State Elimination Method • Many final states are converted to one final
• There are the following rules which are applied for State Elimination Method as given below FA

1. There is no incoming edge to initial state

q1 q1
• We create new initial state ɛ
new initial state ɛ
• Example: ɛ q0 q2 q0 q2 qf
q0 q1 qi q0 q1 ɛ
2. There is no outgoing edge from final state. q2 q2

q0 q1 q2 q0 q1 q2 4. Other than initial state and final state, eliminate other states one by one.
ɛ
qf
87 88
 Cont’d … Cont’d …
a
Note: the following rules are also used. Example 1: b
q0 q2
1. Union 2. Concatenation c
a solution: RE = (a + b + c)
a b
q0 q1 q2
q0 q2 Example 2:
b b
RE = (ab)
RE = (a + b) a c ab*c
q0 q1 q2 q0 q2

a
3. Closure solution: eliminate state q1
q2 RE = ab*c

RE = (a)*
89 90

Cont’d … Cont’d …
0 b c
Example 3: Example 4: d
q0 q1 q0 q1 q2
1 a
1. Add new initial state and new final state as follow: 1. Using the rule 1 add initial state
0 b c
ɛ ɛ ɛ d
q2
qi q0 q1 qf qi q0 q1
a
1
2. Eliminate state q0 as follow: 2. Eliminate state q0 as follow:
(01)* c
d
qi b q1 q2
0 ɛ
qi q1 qf ba*
3. Eliminate state q1 as follow: 3. Combining c and ab*
(c+ba)*
0(01)* So, RE = 0(01)* d
qi qf qi b q1 q2
91 92
 Cont’d …
Activity 3
4. Eliminate state q1

1. Find regular expression for the following DFA:

b(c+ba)*d
qi q2

2. Find regular expression for the following DFA:

So, RE = b(c+ba)*d

3. Find regular expression for the following DFA:

93 12/8/2022 94

Conversion of Regular Grammars to Finite Automata(FA) Cont’d …

Example 1: Consider the following grammar
• Number of states in the automata will be equal to number of non-terminals plus one.
S → 0S | 0A | 1B | 0 | 1
• Each state in automata represents each non-terminal in regular grammar. A → 0A | 1A | 1B |1
• Additional state will be Final State. B → 0A | 1S
Transition of Automata • Solution: using the above given 3 rules, we can construct the follow FA.
1. For every production A → aB, make ƃ(A, a) = B { make an edge labelled a from A to B } 0 0,1
a
• Example: A B
0
2. For every production A → a, make ƃ(A, a) = Final State(FS) S A
a 1
• Example: A FS 0, 1
1 1
3. For every production A → ɛ, make ƃ(A, ɛ) = A will be Final State. 1
• Example: A C
B
0
95 96
 Cont’d … Cont’d …
Example 2: Consider the following grammar. Trace the transition to accept the Example 3: Consider the following grammar
string abb. • Solution: using the above given 3 rules, we can construct S → 0S | 0A | 1|0
A → 0S | 1B |1
the follow FA. B → 1S | 0A
S → abA
S→B b • Solution: using the above given 3 rules, we can construct the follow FA.
S → baB b 0
S a A 0
S→ɛ
A → bS 0
b ɛ S A
B → aS a ɛ b
A→b 0,1
1 1
1 1
C
a
B C
B 0
97 98

Activity 4 Conversion of Finite Automata to Regular Grammar

• Converting a DFA, defined by M = (Q, Σ, δ, q0, F), to a regular grammar, defined
1. Convert the following Regular grammar to Finite Automata. by G = (V, T, S, P) is straight forward.
A. S → aB | bA
B. S → AB | aaB C. S →A • The rules are summarized below:
S → aS | bAA | a A → a | Aa A → aB | Ad
B→b B→b 1. The start symbol of the grammar is q0, the non-terminal corresponding to the
B → bS | aBB | b
C→g start state of the DFA.
D.
S → AB|ε E. S → AB | aaB F. 2. For each transition from state qi to state qj on some symbol ‘a’, create a
S → A1B
A → aB A → a | Aa A → 0A | ε production rule of the form: qi → aqj
B → Sb B→b B → 0B |1B | ε
3. For each state qi of the DFA which is a final state, create a production rule of the
form: qi → ɛ.

12/8/2022 99 100
 Cont’d … Cont’d …
• Example 1: Convert the following FA to Regular Grammar. • Example 3: Convert the following FA to Regular Grammar.
a
Solution: Solution:
a, b b a b S → bR
S R B FS
a S → aA R → aR
S A A → aA | bA | ɛ • Note : FS indicates Final State R → aB
• Example 4: Convert the following FA to Regular Grammar. B→b
A → 0B
• Example 2: Convert the following FA to Regular Grammar. 1 0,1 A → 1A
Solution: 0 B 0 0 D B → 0C
A C
1 1 0,1 B → 1A
q1 → 1q1 | 0q2 1 C → 0D
0 0 C→0
q1 q2 q3 q2 → 1q2 | 0q3 1 C → 1A
D → 0D
q3 → 0q3 | 1q3 | ɛ • Solution: D→0
D → 1D
101 102
D→1

Activity 5 Connection Between Regular Expression and Regular Languages

• Regular Languages: A language is regular if it can be expressed in terms of regular
1. Convert the following Finite Automata to Regular grammar.
expression.
A. 0 0,1 • Regular language is a language that can be defined by regular expression.
0,1 1 • Regular expression (a+b)* denotes the language the set of all strings over the alphabet{a,
A B C
b}.
1
• Example 1: r= (a+b)*(a+bb)

1 0 0 L( r) = {a, bb, aa, abb, ba, bbb, …}

0,1
0 0 1 • The first part, (a+b)*, stands from any string of a’s and b’s.
B. A B C D
• The second part, (a+bb) represents either an a or a double b.
1 • Consequently, L(r) is the set of all strings on(a, b), terminated by either an a or a double b.
12/8/2022 103 104
 Cont’d …
Applications of RE
Example 2: r=(aa)*(bb)*b • Regular expressions are useful in a wide variety of text processing tasks, and more

• L(r) = {b, aab, bbb, aabbb, aaaabbbbb } generally string processing, where the data need not be textual.

• denotes the set of all strings with an even number of a’s followed by an odd • Common applications include :
• lexical analysis and syntax analysis
number of b’s.
• data validation
• L(r) = {a2nb2m+1 : n≥0,m≥0}
• data scraping (especially web scraping)
Example 3: For ∑ = {0, 1} give regular expressions corresponding to regular
• data wrangling
language. • simple parsing
• L(r) = {w ϵ ∑*: w has at least one pair of consecutive zeros} • the production of syntax highlighting systems, and many other tasks.
• L = {00, 100,00111, 101000, 101001100, …..} • While regexps would be useful on Internet search engines, processing them across the

• r = (0+1)*00(0+1)* entire database could consume excessive computer resources depending on the complexity
105 and design of the regex. 106

Context Free Languages

• In formal language theory, a language is defined as a set of strings of symbols

that may be constrained by specific rules.
• Similarly, the written English language is made up of groups of letters (words)

Chapter 3: Context Free Languages separated by spaces.

• A valid (accepted) sentence in the language must follow particular rules, the
grammar.
• A context-free language is a language generated by a context-free grammar.
• They are more general (and include) regular languages.
• The same context-free language might be generated by multiple context-free
grammars.
12/8/2022 AU 107 12/8/2022 AU 108
 Cont’d … Cont’d …
• The set of all context-free languages is identical to the set of languages that are 3. A finite set P of production rules of the form A → w, where A ϵ V is a non
accepted by pushdown automata (PDA). terminal and w ϵ (ΣUV)∗ is a string of symbols and variables.
• Here is an example of a language that is not but is context-free language: 4. A starting non-terminal, typically denoted S.
L=anbn , n>0 • For example, the following eight production rules describe a context free
• This is the language of all strings that have an equal number of a’s and b’s. grammar with terminals Σ = {0,1} and non-terminals V = {S, A, B, C}:
• Formally, a language is context-free if and only if it has a certain type of
recursive description known as a context-free grammar, which is a structure
with the following components:
1. A finite set Σ, whose elements are called symbols or terminals.
2. A finite set V from Σ, whose elements are called non-terminals.
12/8/2022 AU 109 12/8/2022 AU 110

Cont’d …
• Normally we write grammars more compactly by combining the right sides of all rules Context Free Grammars (CFG)
for each non-terminal into one list, with alternatives separated by vertical bars. • CFG stands for Context Free Grammar.

• For example, the previous grammar can be written more compactly as follows: • It is a formal grammar which is used to generate all possible patterns of string in a given
Formal Language.
• Context free grammar G can be defined by four tuples as:
G = (V, T, P, S)
• G is grammar, which consists , of a set of production rule.
➢ It is used to generate the string of a language
• T is the finite set of terminal symbol
➢ It is denoted by lowercase letters.
• V is the finite set of non-terminal symbol
➢ It is denoted by capital letters
12/8/2022 AU 111 12/8/2022 AU 112
 Cont'd…. Cont'd….
• P is set of production rules, • Example1: Construct CFG for the language having any number of a’s over the set ∑
• which is used for replacing non-terminal symbols (on left side of = {a}.
production) in a string with other terminals or non-terminal symbols(on • Solution: ∑ = {a}.
right side of production). L ={ε, a, aa, aaa, aaaa, aaaaaa, ….}
• S is the start symbol used to derive the string. R.E = a*
Derive input string “aaaaaa”
• Example: Production rule S→aS S→aS
S → AS • Start symbol(S) = S
aaS S→aS
S → aS
S → b|aB • Non terminal symbols (V) = A, B aaaS S→aS
S→ε
aaaaS S→aS
• Terminal symbols (T) = b, a
aaaaaS S→aS
• Number of Production rules(P) = 3 aaaaaaS S→aS
aaaaaa S→ ε
12/8/2022 AU 113 12/8/2022 AU 114

Cont'd…. Cont'd….
• Example2: Construct CFG for the language L = { wcwR where w є (a,b)* . • Example3: Construct CFG for the language L = {anb2n where w є (a,b)* .
• Solution: L ={aacaa, bcb, abcba, abbcbba, ….} w = ab • Solution: L =anb2n where n>=1
the grammar could be: wR = ba • L ={abb, aabbbb, aaabbbbbb, ….}
w = abb
wR = bba the grammar could be:
Derive input string “aaabbbbbb”
Derive input string “abbcbba” S → aSbb S→aSbb S→aSbb
S → aSa S→aSa S→bSb S→aaSbbbb S→abb
S→abSba S→bSb S → abb
S → bSb S→ aaabbbbbb
S→abbSbba S→c
S→c S→abbcbba

12/8/2022 AU 115 12/8/2022 AU 116

 Parsing and Derivation Tree
Activity 1
• Parsing of grammar is also called derivation.
• Derivation or Parsing is a sequence of production rules.
1. Find context free grammar for the language L = { anbm : n ≠ m}.
• It is used to get the input string through these production rules.
2. Find context free grammar for the language L = { anbm : n < m}.
• During parsing, we have to take two decisions:
3. Find context free grammar for the language L = {wwR :w {a, b}∗.} – We have to decide the non-terminal which is to be replaced
4. Find context-free grammars for the languages L= anbn, n is even.} – We have to decide the production rule by which the non-terminal will be replaced.

5. Find context-free grammars for the languages L = anbn, n is odd. • Now we have two options to decide which non-terminal to be placed with production

6. Find context-free grammars for language L = {a2nbm : n ≥ 0, m ≥ 0}

rule. S → aABb
1. Leftmost derivation (LMD)
LMD RMD
2. Rightmost derivation (RMD)
12/8/2022 117 12/8/2022 AU 118

…Cont’d …Cont’d
2. Derivation Tree (Parse Tree) The parse tree contains the following properties:
• It is a graphical representation for the derivation of the given production rules • The root node is always a node indicating start symbols.
for a given CFG. • The derivation is read from left to right.
• It is simple way to show how the derivation can be done to obtain some string • The leaf node is always terminal nodes.
from a given set of production rules. • The interior nodes are always the non-terminal node.
• It is also called as parse tree. • Example : Given Production rule: Input string : a*b+c
• Parse tree follows the precedence of operators. E→E+E E
E→E*E E
• The deepest subtree traversed. E→E+E E→a E
E → a|b|c +
→a*E E→E+E
• So, the operator in the parent node has less precedence over operator in the E
→a*b+E E→b * E c
subtree. →a*b+c E→c
a b
Derivation of string a* b + c
12/8/2022 AU 119 12/8/2022 AU 120
Parse Tree
 …Cont’d …Cont’d
1. Leftmost derivation 2. Right Most Derivation
• In the leftmost derivation, the input is scanned and replaced with production rule from • In the Right Most Derivation, the input is scanned and replaced with production rule
left to right . from right to left .
E→E+E E→E+E
• So, we have to read input string from left to right. E→E–E • So, we have to read input string from right to left. E→E–E
• Example : Production rules and input string: a – b + a E → a|b • Example: Production rules and input string: a – b + a E → a|b

• Derivation Tree • The right most derivation is: • Derivation Tree

• The leftmost derivation is:
E E→E–E E→E+E E
E→E+E E→E–E
E→E–E+E E→a E
E→E–E+E E→a E E E
+ E→E–E+a E→b -
E→a–E+E E→b
E E E→E–b+a E→a E
E→a–b+E E→a - a a E
E→a–b+a +
E→a–b+a
a b a
12/8/2022 AU 121 12/8/2022 AU
b 122

…Cont’d …Cont’d
• Example1: Derive input string abb for left most derivation and right most • The right most derivation is:
• Derivation Tree is:
derivation using CFG given by below: S → AB|ε S → AB B → Sb
A → aB S → ASb S→ε
• The left most derivation is: B → Sb S
S → Ab A → aB
S → aBb B → Sb B
S → AB A → aB • Derivation Tree is: A
S → aSbb S→ε
→ aBB B → Sb S a
S → abb B S b
→ aSbB S→ε
→ abB B → Sb B
A
b ɛ
→ abSb S→ε S
→ abb a B S b
ɛ
b ɛ
S

12/8/2022 AU ɛ 123 12/8/2022 AU 124

 …Cont’d …Cont’d
• The right most derivation is:
• Example2: Derive input string 00101 for left most derivation and right most • Derivation Tree

derivation using CFG given by below: S → A1B S → A1B B → 0B S

A → 0A|ε S → A10B B → 1B
• The left most derivation is: S B → 0B|1B|ε S → A101B B→ε A 1 B
• Derivation Tree S → A101 A→ 0A
→ A1B A → 0A S → 0A101 A → 0A 0 A 0 B
S → 0A1B A → 0A S
S → 00A101 A→ε
S → 00A1B A→ε S → 00101 0 1 B
A B A
S → 001B B → 0B 1
S → 0010B B → 1B 0 ɛ
A 0 B ɛ
S → 00101B B→ε
0 A 1 B

ɛ ɛ
12/8/2022 AU 125 12/8/2022 AU 126

…Cont’d …Cont’d
• Example3: Derive input string aabbabba for left most derivation and right most • Example4 : Consider the grammar:

derivation using CFG given by below: Obtain derivation tree of arithmetic expression
S → aB|bA E→E+E
A → a|aS|bAA E→E*E (i) id*id+id
B → b|bS|aBB E→ (E) (ii) (id+id)*id
• The left most derivation is:
E→ id (ii) id*id+id
• The right most derivation is: S
S → aB B → aBB E
S → aaBB B→b → aB B → aBB
Solution: (i) id*id+id
S → aab B → bS S → aaBB B → bS E + E
S → aabbS S → aB S → aaBbS S → bA E→E+E E→E*E
S → aabbaB B → bS S → aaBbbA A→ a E→ E*E + E E→ id E E
* id
S → aabbabS S → bA S → aaBbba B → bS E→ id*E + E E→ id
S → aabbabbA A→ a S → aabSbba S → bA E→ id*id + E E→ id id id
S → aabbabba S → aabbAbba A→a E→ id*id + id
12/8/2022 AU
S → aabbabba 127 12/8/2022 AU
Parse Tree for string id*id+id 128
 …Cont’d
Activity 2
(ii) (id+id)*id
1. Perform leftmost and rightmost derivation for the grammar for the string
E S → aAB
E→E*E E → (E) abbbb and finally construct parse tree for both derivation. A → bBb
E→ (E)*E E→ E + E E E B →A|ε
* 2. Perform leftmost, rightmost derivation for the grammar and construct a
E→ (E + E)*E E→ id
E→ (id + E)* E E→ id ( ) derivation tree for the string aabbbb with the grammar: S → AB | ε
E→ (id+id )* E E→ id E id A → aB
E→ (id+id )* id 3. Consider the grammar with productions. B → Sb
E E S → aaB
+
A → bBb | ε
id id B → Aa

Show that the string aabbabba is not in the language generated by this grammar.

12/8/2022 AU 129 12/8/2022 130

Sentential Forms Ambiguous Grammar

• Derivation from the start symbol produce strings that have a special role. • A grammar is said to be ambiguous if there exists :

• These is called sentential form. – more than one left most derivation or

• That is, given a CFG G = (V,T, P, S) , if where α ϵ (VƲT)*,then α is – more than one right most derivation or
– more than one parse tree for given input string.
a sentential form.
• If the grammar is not ambiguous, then we call unambiguous.
• If where α ϵ (VƲT)*, then α is a left-sentential form.
• If the grammar has ambiguity, then it is not good for compiler construction.
• If where α ϵ (VƲT)*, then α is a right-sentential form.
• No method can automatically detect and remove the ambiguity, but we can remove
ambiguity by re-writing the whole grammar without ambiguity.

12/8/2022 AU 131 12/8/2022 AU 132

 …Cont’d …Cont’d
• Example1: Let us consider a grammar with production rule. • Example2 : Check whether the given grammar is ambiguous or not.
E→E+E
E→ E * E E→E+E
E→ ε|0|1|2|… 9 E→E-E
E→ id
• For the string 3 * 2 + 5, the above grammar can generate two parse tree by leftmost
• For the string id+id-id, the above grammar can generate two parse tree by
derivation.
E leftmost derivation.
E E→E-E E
E→E+E E E→ E+ E - E E E
E E + E E→ id + E E→ id + E - E -
E * • Since there are two parse tree for single E
E
E E * E input string, the grammar is ambiguous. E→ id + E - E + E→ id + id - E E E
3 E+ 5 E→ id + id - E E E→ id + id - id + id
id E -
2 5 3 2 E→ id + id - id id id
12/8/2022 AU 133 12/8/2022
id id AU 134

…Cont’d
Activity 3 S → AB | aabB
• Example3 : Check whether the given grammar is ambiguous or not.
1. Show that the following grammar is ambiguous or not. A → a| Aa
B→b
S → aSb|SS 2. Construct an unambiguous grammar equivalent to the grammar
S→ε

• For the string aabb, the above grammar can generate two parse tree by S → AB | ε
A → aB
leftmost derivation. S B → Sb
S
S S a b
b S 3. Perform leftmost, rightmost derivation for the grammar and construct a
ε a S a derivation tree for the string aabb, with the grammar:
S b
a S b
S = aSb | SS
ε
ε S=
12/8/2022 AU 135 136
 Unambiguous Grammar …Cont’d
2. If the right associative operates(^) is used in the production rule, then apply right
• A grammar can be unambiguous if the grammar does not contain ambiguity:
recursion in the production rule.
omore than one left most derivation or
• Right recursion means that the rightmost symbol on the left side is the same as
omore than one right most derivation or
the non-terminal on the right side. X → aX
omore than one parse tree for given input string.
Example1: Consider a grammar G is
• To convert ambiguous grammar to unambiguous grammar, we apply the following
S → AB|aaB
rules:
A → a|Aa
1. If the left associative operators(+,-,*,/) are used in the production rule, then B→b
• Determine whether the grammar G is ambiguous or not.
apply left recursion in the production rule.
• If G is ambiguous, Construct an unambiguous grammar equivalent to G.
• Left recursion means that the leftmost symbol on the right side is the same as
the non-terminal on the left side. X → Xa
12/8/2022 AU 137 12/8/2022 AU 138

…Cont’d …Cont’d
• Check for the string aab, the above grammar can generate two parse tree by • Example2: Consider a grammar G is
leftmost derivation S S S→S+S
A S → id
B a a B
A • Determine whether the grammar G is ambiguous or not.
a
b b • If G is ambiguous, Construct an unambiguous grammar equivalent to G.
a S → AB • Let us consider string id+id+id
S → AB|aaB A → Aa|a
A → a|Aa B→b S S
B→b • unambiguous grammar by S S S • This is ambiguous grammar
+ S +
• This is Ambiguous grammar applying rules S S S
+ id S +
id id id id
12/8/2022 AU 139
id
12/8/2022 AU 140
 …Cont’d …Cont’d
S → S + id S S→S+S
S → id S S→S*S
+ id S → id
S
+ id
id S→S+T
• Example3: Consider a grammar G is S→T
S T→T*F
S → S + S | S * S | id S T→F
S
• Parse tree for id+id*id * S F → id
S S S S
+ +
id S * S id
id • This is unambiguous grammar.
id
id id
• This is Ambiguous grammar
12/8/2022 AU 141 12/8/2022 AU 142

Activity 4 Simplification of Context Free Grammar

1. Construct an unambiguous grammar equivalent to the grammar given below: • The properties to reduce the grammar

S → AB | aabB 1. Each non-terminal and terminal of G appears in the derivation of some word in L.
A → a| Aa 2. There should not be any production as X → Y where X and Y non terminals.
B→b
3. If ε is not in language L then there need not be the production X → ε
2. Show that the following grammar is ambiguous or not : S → aSbS |bSaS | ε
Reduce Grammar by:
3. Show that the following grammar is ambiguous or not : S →aSb |SS | ε
i. Removal of Useless Symbols
4. Is the string aabbababb in the language generated by the grammar given
ii. Elimination of ε – Production
below? if so show . S → aSS|b iii. Removal of Unit Production

12/8/2022 143 12/8/2022 AU 144

 …Cont’d …Cont’d
1. Removal of Useless
• To remove this useless production A → aA, we will first find all the variables
• A symbol can be useless if it does not appear on the right hand side of production
which will never lead to terminate a string such as variable ‘A’.
and it does not take part in the derivation of any string.
• Then we will remove all the productions in which variable ‘A’ occurs.
• That symbol is useless.
• After removing all useless symbols the above grammar will look like below:
• Example: T → aaB | abA | aaT input string aabba
T → aaB | aaT
A → aA B → ab | b
B → ab | b
C → ad 2. Elimination of ε - Production
• Here, variable ‘C’ is useless because it never occur in derivation of string aabba.
• The productions of type S → ε are called ε productions.
• So, we can eliminate the C production rule.
• These type of production can only be removed from those grammars that do not
• A → aA is also useless because there is no way to terminate.
generate ε.
• It can never produce a string.
12/8/2022 AU 145 12/8/2022 AU 146

…Cont’d
…Cont’d
Step1: First find out all nullable non-terminals which derive ε or not. Solution: while removing ε production, we are deleting the rule X → ε and Y → ε
Step2: For each production A → a, construct all production A → x, where x is • to preserve the meaning of CFG, we are actually placing ε at right hand side
obtained from ‘a’ by removing 1 or more non-terminals from step1. where ever X and Y appear.
Now combine result of step 2 with the original production and remove ε production. S → XYX • If first X at right hand side is ε , then S → YX
• if last X at right hand side is ε. then S → XY
S → XYX
• if Y = ε , then S → XX
X → 0X| ε
Y → 1Y| ε • if both Y and X are ε, then S → X
• if both X are replace with ε, then S → Y
• Now,
S → XY|YX|XX|X|Y
• Let us consider, X → 0X| ε if we replace ε at right hand side for X, then X → 0
X → 0X| 0
12/8/2022 AU 147 12/8/2022 AU 148
 …Cont’d …Cont’d
• Similarly, Y → 1Y| 1 Step3: repeat step1 and step2 until all unit productions are removed.
• Finally, the CFG after the removal of ε production as : S → 0A | 1B | C
S → XY | YX | XX | X | Y A → 0S | 00
X → 0X | 0 B→1|A
Y → 1Y | 1 C → 01
3. Removing Unit Production solution: i) S → C is a unit production.
• The unit productions are the productions in which one non-terminal gives another But while removing S → C we have to consider what C gives. So we can add a rule
non-terminal. to S. S → 0A | 1B | 01
• To remove unit production ii) B → A is also a unit production S → 0A | 1B | 01
Step1: to remove X → Y , add production X→a to the grammar rule, Y → a B → 1 | 0S | 00 A → 0S | 00
• Finally, we write CFG without unit productions as: B → 1 | 0S | 00
occurs in the grammar. C → 01
Step2: now delete X → Y from the grammar.
AU 149 12/8/2022 AU 150

…Cont’d
Activity 4
1. Simplify the following grammar using Removing Useless Productions Rules: 3. Simplify the following grammar using Removing Unit-Productions Rules:

i. S→A S → aS | A | C i. S → Aa|B
A → aA | ɛ ii. A → a | bc S → aA | aBB | A
A→ a
B → bA B → bb|A ii. A → aaA | ɛ
B → aa
B → bB | bbC
B → aCb C→B
2. Simplify the following grammar using Removing ɛ-Productions
ii. S → ABaC
i. S → aS1b A → BC
S1 → aS1b | ɛ B→b|ɛ
C→D|ɛ
D→d
12/8/2022 151 12/8/2022 152
 …Cont’d
Chomsky's Hierarchy of Grammars 1. Type 0 Grammar:
• Chomsky Hierarchy represents the class of languages that are accepted by the different • Type 0 grammar is known as Unrestricted grammar.
machine. • There is no restriction on the grammar rules of these types of languages.
• The category of language in Chomsky's Hierarchy is as given below: • These languages can be efficiently modeled by Turing machines.
1. Type 0 known as Unrestricted Grammar. • For example: bAa → aa
2. Type 1 known as Context Sensitive Grammar. S→s
3. Type 2 known as Context Free Grammar. 2. Type 1 Grammar:
4. Type 3 known as Regular Grammar. • Type 1 grammar is known as Context Sensitive Grammar.

• Therefore every language of type 3 is also of type • The context sensitive grammar is used to represent context sensitive language.

2, 1 and 0. • The context sensitive grammar follows the following rules:

• Similarly, every language of type 2 is also of type o The context sensitive grammar may have more than one symbol on the left hand
112/a8n/ 20d22type 0, etc. AU 153 12/8/2022 side of their production rules. AU 154

…Cont’d …Cont’d
o The number of symbols on the left-hand side must not exceed the number of • Context free languages are the languages which can be represented by the context
symbols on the right-hand side. free grammar (CFG).
o The rule of the form A → ε is not allowed unless A is a start symbol. • Type 2 should be type 1.
• It does not occur on the right-hand side of any rule. • The production rule is of the form: A→α
o The Type 1 grammar should be Type 0. • Where A is any single non-terminal and α is any combination of terminals and
• In type 1, Production is in the form of V → T non-terminals. A → aBb
• Where the count of symbol in V is less than or equal to T. • For example: A→b
B→a
• For example: S → AT 4. Type 3 Grammar:
T → xy
A→a • Type 3 Grammar is known as Regular Grammar.
3. Type 2 Grammar: • Regular languages are those languages which can be described using regular
• Type 2 Grammar is known as Context Free Grammar(CFG). expressions.
12/8/2022 AU 155 12/8/2022 AU 156
 …Cont’d
• These languages can be modeled by NFA or DFA.
• Type 3 is most restricted form of grammar.
• The Type 3 grammar should be Type 2 and Type 1.
• Type 3 should be in the form of:

V → T*V / T* Chapter 4: Push Down Automata(PDA)

• For example:

A → xy

12/8/2022 AU 157 12/8/2022 AU 158

Cont’d …
Push Down Automata(PDA)
• To read an element into the stack, the top elements must be popped off and are lost.
• Pushdown automata is a way to implement a CFG in the same way we design DFA
• A PDA is more powerful than FA.
for a regular grammar.
• Any language which can be acceptable by FA can also be acceptable by PDA.
• A DFA can remember a finite amount of information,
• PDA also accepts a class of language which even cannot be accepted by FA.
• But a PDA can remember an infinite amount of information.
• Thus PDA is much more superior to FA.
• Pushdown Automata is a finite automata with extra memory called stack .
• Basically a PDA is :
• The addition of stack is used to provide a last-in-first-out memory management
" Finite state machine " + " a stack ".
capability to Pushdown automata.
• PDA has three components:
• Pushdown automata can store an unbounded amount of information on the stack.
1. An input tape: The input tape is divided in many cells or symbols.
• It can access a limited amount of information on the stack.
• The input head is read-only and may only move from left to right, one symbol at
• A PDA can push an element onto the top of the stack and pop off an element from the
a time.
top of the stack.
159 12/8/2022 AU 160
 Cont’d … Cont’d …
2. Finite control: The finite control has some pointer which points the current symbol • The diagram that shows a transition in a PDA from a state q1, to state q2, labeled
which is to be read. as a, b →c. Stack top
3. Stack: The stack is a structure in which we can push and remove the items from one symbol Push
i/p symbol symbol
end only.
• It has an infinite size.
• In PDA, the stack is used to store the items temporarily.
a, b / c
q1 q2
• At state q1, if an i/p string is 'a' is encounter and top symbol of the stack is 'b'
then we pop 'b' , push 'c' on top of the stack and move to state q2.

12/8/2022 AU 161 12/8/2022 AU 162

Cont’d …
Terminologies Related to PDA
Formal Definition of PDA:
• A PDA can be formally described as 7-tuples (Q, ∑, Γ, δ, q0, Z, F). 1. Instantaneous Description(ID)
o Q is finite number of states. • ID is an informal notation of how a PDA computes an input string
o ∑ is input alphabet. and make a decision that string is accepted or rejected.
o Γ is a stack symbol which can be pushed and popped from the stack
• The instantaneous description (ID) of a PDA is represented by a
o q0 is the initial state (q0 ϵ Q)
triplet (q, w, s )
o Z is a start symbol which is in Γ.
o F is set of accepting states (F ϵ Q) o q is a current state
o δ is mapping function which is used for moving from current state to o w is unconsumed input
next state. o s is the stack contents
12/8/2022 AU 163 12/8/2022 AU 164
 Cont’d …
2. Turnstile Notation: PDA Acceptance

• sign describes the turnstile notation and represents one move. • There are two different ways to define PDA acceptability.

• It is used for connecting pairs of ID’s that represent one or many 1. Final State Acceptability
• A PDA accepts a string when, after reading the entire string, the PDA in final
moves of a PDA.
state.
• * sign describes a sequence of moves.
• From the starting sate, we can make moves that end up in a final state with
• Example: (p, b,T ) (q, w, a)
any stack values.
• While taking a transition from p to q, the input symbol ' b ' is consumed
• The stack values are irrelevant as long as we end up in a final state.
and top of the stack 'T' is represented with a new string 'a '.
• For a PDA ((Q, ∑, S, δ, q0, I, F) the language accepted by the set of final sates
F is L(PDA) ={w | (q0, w, I) * (q, ɛ, x), q ϵ F} for any input stack string x.
12/8/2022 AU 165 AU 166

Cont’d … Cont’d …
2. Empty Stack Acceptability • Example1: Construct PDA that accepts L = {anbm | n ≥ m, m ≥1}in final state.
• Here a PDA accepts a string when , after reading the entire string, the PDA • Solution: simple logic at least one b is there and 2 a's should be there.
has emptied its stack. L={aab, aaabb, aaaabbb. ….}
• Now we can write the ID as
• For a PDA (Q, ∑, S, δ, q0, I, F) the language accepted by Empty Stack a, a|aa follows:
a, z0 |az0 δ(q , ε, Z ) = (q , Z )
L(PDA) ={w | (q 0, w, I) * (q, ɛ, ɛ), q ϵ Q} for nay input string x.
ɛ, ɛ|z0 (b, a| ɛ) 0 0 0 0

Empty stack δ(q0, a, Z0) ) = (q0, a)

b, a| ɛ ɛ, a|z0
q0 q1 q2 δ(q0, a, a) = (q0, aa)
δ(q0, b, a) = (q1, ε)
Extra a in the stack
δ(q1, b, a) = (q1, ε)
δ(q1, ε, a) = (q2, aZ0)

12/8/2022 AU 167 12/8/2022 AU 168

 Cont’d … Cont’d …
• Now we will simulate this PDA for the input string "aaabb". • Example 2 : Design a PDA for accepting a language {anb2n | n>=1} in empty stack

δ(q , aaabb, Z ) push a acceptability

0 0
• Solution: strings belong to this language are {abb, aabbbb, aaabbbbbb,….}
δ(q0, aabb, aZ0) push a
• In this language, n number of a's should be followed by 2n number of b's.
δ(q0, abb, aaZ0) push a • Hence, we will apply a very simple logic, and that is if we read single 'a', we will
δ(q0, bb, aaaZ0) push a push two a's onto the stack.
δ(q0, bb, aaaZ0) pop a • As soon as we read 'b' then for every single 'b' only one 'a' should get popped from
δ(q , b, aaZ ) pop a the stack. • For a single a push 2 a’s on the aabbbb a b
1 0 top of the stack.
(q , ε, aZ ) • If you read input b pop 1 a from a b
1 0 the top of the stack a b
δ(q2, a) Accept a b
• After popping each a for corresponding
12/8/2022 AU 169 12/8/2022
each b, the stack is empty
AU
Z0 170

Cont’d … Cont’d …
• So, the PDA diagram will be constructed as follows: • Now we will simulate this PDA for the input string "aabbbb".

a, a|aa δ(q0, aabbbb, Z0)

a, z0 | ɛ b, a|ɛ δ(q , abbbb, aaZ ) push 2 a on stack
0 0
b, a|ɛ ɛ, z0|z0 q2 δ(q , bbbb, aaaaZ ) push 2 a on stack
q0 q1
0 0

• Now we can write the Instantaneous Description(ID) as follows: δ(q1, bbb, aaaZ0) pop 1 a from the stack
δ(q0, a, Z0) = (q0, aaZ0) δ(q1, bb, aaZ0) pop 1 a from the stack
δ(q0, a, a) = (q0, aaa) δ(q1, b, aZ0) pop 1 a from the stack
δ(q0, b, a) = (q1, ε)
δ(q1, ε , Z0)
δ(q0, b, a) = (q1, ε)
δ(q , ε, Z ) = (q , Z ) δ(q2, ε)
1 0 2 0

171 172
 Cont’d …
Deterministic Push Down Automata (DPDA)
• So, for a deterministic PDA, there is at most one transition possible in any
• The DPDA is a variation of pushdown automata that accepts the deterministic
combination of state, input symbol and stack top.
context-free languages.
Formal Definition of Deterministic PDA
• A language L(A) is accepted by a DPDA if and only if there is a single
• A Deterministic PDA is 5 tuple:
computation from the initial configuration until an accepting one for all strings
M = (Σ,Γ,Q,δ,q)
belonging to L(A). Σ - It is a finite set which does not contain a
• It is not as powerful as non-deterministic finite automata. empty symbol,

• That's why it is less in use and used only where determinism is much easier to Γ - a finite set of stack alphabet,
Q - set of states,
implement.
q - start state,
• A PDA is said to be deterministic if its transition function δ(q, a, X) has at most
δ - a transition function, denoted as -
one member for a Σ U {ε}. δ : Q × (Σ {ɛ}) × Γ → Q × {N,R} × Γ
12/8/2022 AU 173 12/8/2022 AU 174

Non-Deterministic Push Down Automata (NPDA) Cont’d …

• A non-deterministic pushdown automaton is a 7-tuple.
• A non-deterministic PDA is used to generate a language that a deterministic
NPDA = (Q, Σ, Γ, δ, q0, Z, F)
automata cannot generate.
where
• It is more powerful than a deterministic PDA.
• So, a push down automata is allowed to be non-deterministic. • Q- It is the finite set of states,
• Σ - finite set of input alphabet,
• Γ - finite set of stack alphabet,
• δ - transition function,
• q0 - initial state,
• Z - stack start symbol,
• F - finite states.
12/8/2022 AU 175 12/8/2022 AU 176
 Cont’d …
Pushdown Automata and Context-Free Languages
• Example: Construct NPDA for the language L={anb3n | n>=1} .
Context Free Language
• Solution: string of the language are w={abbb, aaabbbbbb, …..}
• A language is context free if there exists a context free grammar that can generate
it.
a, a|aaaa • Example1: The language L=anbn, n≥0, is generated by a grammar whose rule are:
a, z0|aaaz0 b, a|ɛ
b, a|ɛ
S → aSb
ɛ, z0|z0 ɛ, z0|z0 S→ε
q0 q1 q2
• Example2: The language L=wwR, n≥0, is generated by a grammar whose rule are:
S → aSa
S → bSb
S→ε
• A language is context free if and only if it is accepted by pushdown automata.
12/8/2022 AU 177 12/8/2022 178

Cont’d …
Properties of Context Free Languages • The corresponding grammar G will have the additional production S → S1 | S2.
• Context-free languages are closed under:
2. Concatenation
1. Union
• If L1 and L2 are context free languages, then L1L2 is also context free language.
2. Concatenation
• Example
3. Kleene Star operation • Union of the languages L and L , L = L L = { anbncmdm }
1 2 1. 2
1. Union
• The corresponding grammar G will have the additional production S → S1S2
• Let L1 and L2 be two context free languages.
3. Kleene Star
• Then L1 L2 is also context free language.
• If L is a context free language, then L* is also context free language.
• Example
• Example
• Let L1 = { a nbn , n > 0}. Corresponding grammar G1 will have P: S1 → aAb|ab
• Let L1 = { anbn , n ≥ 0}. Corresponding grammar G will have P: S → aAb| ε
• Let L2 = { cm d m , m ≥ 0}. Corresponding grammar G2 will have P: S 2 → cBb| ε • Kleene Star L1 = { anbn }*
• Union of L 1 and L 2, L = L 1 L2 = { anbn } { cmdm }.
12/8/2022 179 • The corresponding grammar G1 will have additional productions S1 → SS1 | ε
4. If LR is a regular language and if the language L2 is context free, then LR ∩ L2 is
Application of Push Down Automata (PDA)
also Context free language.
• Context-free languages are not closed under:
• For designing the parsing phase of a compiler (Syntax Analysis).
• Intersection − If L1 and L2 are context free languages, then L1 ∩ L2 is not
necessarily context free language.
• For implementation of stack applications.

• Intersection with Regular Language − If L1 is a regular language and L2 is • For evaluating the arithmetic expressions.
a context free language, then L1∩ L2 is not a context free language. • For solving the Tower of Hanoi Problem.
• Complement − If L1 is a context free language, then L1' may not be context
free.

12/8/2022 181 12/8/2022 182

Activity 1

1. Construct PDA for a language L = { anbn : n ≥ 0} and simulate using PDA for the

string aaabbb acceptance or not.

2. Construct PDA for language L = {a2nbm : n ≥ 0, m ≥ 0} and simulate using PDA

for the string aab acceptance or not. Chapter 5:Turing Machine (TM)
3. Construct PDA for language L = {anbmcn : n ≥ 1, m ≥ 1} and simulate using PDA

for the string aabbbcc acceptance or not.

4. Design PDA for Palindrome strings and simulate for the string abaaba

12/8/2022 183 12/8/2022 AU 184

 Cont’d …
Turing Machine(TM) • A Turning machine basically has three main parts:
• A Turning Machine is a very powerful computing machine.
1. Finite State Control - It gives a command to the machine.
• This was invented by Alan Turning in 1936.
2. Infinite Tape - Memory device of machine.
• The basic fundamental of this machine is to build an automaton with finitely
3. Tape Head - It reads and writes a single cell.
many states but unbounded memory.
✓ Is a model of general purpose computer.
✓ Can do everything that a real computer can do.
• The blank # is special symbol # Ɇ ∑.
✓ Uses an infinite tape as its unlimited memory.
• The blank symbol is a special symbol used to fill the infinite tape.
✓ Has a tape head that can read and write symbols and move around on the
tape.
✓ Initially the tape contains only the input string and is blank everywhere else.
AU 185 186

Cont’d … Cont’d …
• Turing machine is has infinite size tape and it is used to accept Recursive
• A Turning machine is a finite automaton that has an infinite memory tape.
Enumerable Languages.
• The tape is divided into many infinite blank cells.
• TM can move in both directions (LEFT or RIGHT)
• Each cell contains a symbol from some finite alphabet.
• Also it doesn't accept ɛ.
• The machine operates on this tape.
• If the string inserted is not in the language, machine will halt in non-final
• It has a tape head that reads and writes a single memory cell at a time.
state.
• It also moves the tape cell left or right at a time.
• After reading an input symbol, it is replaced with another symbol, its internal
• In the turning machine, this tape starts with the input written on it.
state is changed and it moves from one cell to the right or left.
• If the TM reaches the final state, the input string accepted, otherwise
rejected.
AU 187 AU 188
 Cont’d …
The Standard Turing Machine(TM)
• A Turning machine is a 7 tuple:
TM = (Q, Σ, Γ, δ, q0, F, #)

• Where
Q - is the finite set of states,
Σ - finite set of input alphabet,
Γ - tape alphabet,
Note
δ - transition function, Q x Γ -> Q x Γ x {L,R},
• L is move left
q0 - start state, • R is move right
F - accept state (Final State),
# - blank symbol
AU 189 AU 190

Cont’d … Cont’d …

• So, a turning machine has two alphabets, the input alphabet and the tape • States & Transitions

alphabet.
• The tape alphabet contains all symbols that can be written on the tape.
• But, it can contain at least one blank symbol.
• The blank symbol is denoted with #.
• When a Turning Machine receives an input, it can do one of the three things
with the input:
1. Accept the input (halt in final state)
2. Reject the input (halt in non final state)
3. Continue the computing (loop for ever)
AU 191 12/8/2022 RVU 192
 Cont’d … Cont’d …
Determinism
Final States
• Turing Machines are deterministic: for each state there is only one unique
Transitions on each symbol.
• The given TM is deterministic if and only if there should be a transition for
each and every input symbol that we have for each and every state.
Allowed Not Allowed

12/8/2022 RVU 194

Turing Decidable and Turing Acceptable

• The move will be б (q0 , a) → б (q1 , A, R) which means it will go to state q1 ,
• The TM accepts all the language even though they are recursively enumerable.
replaced by ‘a’ by ‘A’ and head will move right as :
• Recursive means replacing same set of rules for any number of time. …. A b a # # ….
• Enumerable means list of elements. q1
• The move will be б (q1 , b) → б (q2 , B, R) which means it will go to state q2 ,
• TM also accepts the computable functions such as addition, multiplication,
replaced by ‘b’ by ‘B’ and head will move right as :
substruction, division, and many more. … A B a # # …
q2
• Example: Construct a TM which accepts the language of "aba".
• The move will be б (q2 , a) → б (q2 , A, R) which means it will go to state q3 ,
• If the tape head is "readout" "aba" string then TM will halt after reading #.
replaced by ‘a’ by ‘A’ and head will move right as :
• Initially state is q0, and head points to ‘a’ as … A B A # # …
q3
…. a b a # # …. • The move will be б (q3 , #) → б (q4 , #, H) which means it will go to state q4 ,
q0 which
12/8/2022is HALT state which is accepted by the TM. 196
12/8/2022 RVU 195
 Cont’d … Cont’d …
• It can be represented by Transition Table • Example1: Construct a TM that recognizes the language L={01*0} over ∑ = {0,1}.
States a b # • Solution: {00, 010, 0110, 01110,…..}
q0 (q1 ,A,R) - -
q1 - (q2 ,B,R) - 1→Y,R
q2 (q3 ,A,R) - -
q3 - - (q4 ,#,H) q0 0→X,R 0→X,R q2 #→#,L Accept
q1
q4 - - -
# → #, R
• It can be represented by Transition Diagram as follows:
1→Y,R 1→Y,R
#→#,R 0→#,0
q0 a→A, R b→B,R a →A, R #→#,H
q1 q2 3 q4
Reject

12/8/2022 RVU 197 12/8/2022 RVU 198

Cont’d … Cont’d …
• Example2: Construct a TM that recognizes the language L={0N1N | N≥1} over ∑ = Y → Y, R Y → Y, L
{0,1}. 0 → 0, R 0 → 0, L
• Solution: {00001111} when N= 4 00 0 011 1 1 # ... q0 0 → X, R 1 → Y, L q2
q1
• Algorithm
1. change '0' to 'X' Y → Y, R X → X, R
2. Move right to the first '1'
# → #, L q3
• If none: reject
3. Change '1' to 'Y' # → #, L

4. Move left to leftmost '0’ repeat the above steps until no more 0’s.
q4
5. Make sure no more 1's also.
12/8/2022 RVU 199 12/8/2022 RVU 200
 Cont’d …

• Example: A Turing machine that accepts the language: L={aa*}

a → a, R

# #, L
q0 q1

12/8/2022 RVU 201

Cont’d … Cont’d …

• Example: Rejection Example

• No possible Transition for input symbol b. Halt & Reject

Halt & Accept

12/8/2022 RVU 204

 Co nt’d Cont’d

• Infinite Loop Example

• A Turing machine for language

• Because of the infinite loop:

✓ The final state cannot be reached

✓ The machine never halts

✓ The input is not accepted

12/8/20 22 RVU 205 12/8/2022 RVU 206

Recursive and Recursively Enumerable Languages Undecidable Problems

• A problem is said to be decidable or solvable if and only if there exist an
• The machine halts if there are no possible transitions to follow.
algorithm that takes as input as instance of the problem and determines
• The Turing Machine may:
whether the answer to that instance is YES or NO.
✓ Halt and accept the input.
• Otherwise the problem is said to be undecidable or unsolvable.
✓ Halt and reject the input
• A problem is said to be undecidable or unsolvable if and only if there exist no
✓ Never halt (loop for ever)
algorithm that takes as input as instance of the problem and determines
• Recursively Enumerable Language whether the answer to that instance is YES or NO.
• There is a TM for a language which accepted a string otherwise not.
• A problem whose language is recursive is said to be decidable or solvable,
• Recursively Language otherwise the problem is undecidable or unsolvable.
• There is a TM for a language which halt every string.
12/8/2022 RVU 207 12/8/2022 RVU 208
 Activity 1
Application of Turing Machine(TM)
• The applications of Turing machine are as follows: 1. Construct Turing machine for the string 'aabb’.

✓ Used to solve the recursively enumerable problem. 2. Construct Turing machine for the language that accepts strigs starting with '0'

and ending with '1' over ∑ ={0,1}.

✓ Used for knowing complexity theory.
3. Design Turing Machine for the language that accepts strings starting with 2
✓ Used for neural networks implementation.
consecutive 'b' s and terminated with a single 'a' over ∑ ={a,b}.
✓ Used in Robotics Applications.
4. Construct Turing Machine for a is smaller than b L = { ambn : m<1}.
✓ Used in the implementation of artificial intelligence(AI).
5. Design Turing Machine for the complement of a string over ∑ ={0,1}.

6. Construct Turing machine for anbncn | n ≥ 1 },

7. Construct Turing machine for unary integer substruction.

12/8/2022 209 210

Computability

Computability

Chapter 6 Computability – The study of what can/cannot be done via purely mechanical means.
Goals of Computability
– Study the properties of classes of solvable and unsolvable problems.
– Solve or prove unsolvable open problems.
Computable Functions
– A function is computable, if it can be computed by a TM.
 Computability … Computability …

– A TM is a very simple machine but logically speaking, TM has all the

power of any digital computer.
– TM may be described as a TM processes an infinite Tape.
– A Turing machine computes a function f : Σ → Σ if, for any input word
w, it always stops in a configuration where f(w) is on the tape.
– The functions that are computable by an effective procedure are those
that are computable by a Turing machine.

Computability … Computability …

Integer Domain
Definition
– A function f is computable if there is a Turing Machine M with a
transition from initial state to final state.
 Computability … Computability …
– The functions that can be implemented by a TM are said to be – Note: The 0 is the delimiter that separates the two numbers.

computable functions.

Computability … Computability …
– Example: Given two integers x and y design a TM that compute x + y.
– Solution: let us consider x=3 and y=2
• x= 3 means 111 in unary representation
• y=2 means 11 in unary representation
• So, x + y =3 + 2 means 11111
Time 4: (q1, 1)=(q1,1,R)
Transition function
Time 5: (q1, 1)=(q1,1,R)
Time 0: (q 0, 1)=(q 0,1,R)
Time 1: (q , 1)=(q ,1,R) Time 6: (q1, #)=(q2,#,L)
0 0
Time 2: (q , 1)=(q ,1,R) Time 7: (q2, 1)=(q3,0,R)
0 0

Time 3: (q0, 0)=(q1,1,R) Time 8: (q3, #)=(q4,#,R)

 Computability … Recursive Functions
– Transition Diagram for the above problem is :
– A recursive function is one that calls upon itself to determine the solution.
– Example: Fibonacci Numbers
1→1,R 1→1,R F(0) = 1, F(1) = 1

#→#,R
F(n+1) = F(n) + F(n-1) for all n > 0
q0 0→1,R #→#,L 1→0,L q3
q1 q2 q4 – Primitive Recursive Functions are defined by starting with some base set
of functions and then expanding this set via rules that create new primitive
recursive functions from old ones.
End!!!
Thank You!!!

Recursive Functions … Recursive Functions …

– The base functions are: 1. Zero (Constant)

2. Successor
1. Zero (Constant)
2. Successor
3. Projection
 Recursive Functions … Recursive and Recursively Enumerable Languages

3. Projection – The machine halts if there are no possible transitions to follow.

– The Turing Machine may:
• Halt and accept the input.
• Halt and reject the input
• Never halt (loop for ever)

Recursive and Recursively Enumerable Languages … Recursive and Recursively Enumerable Languages …

• Recursively Language
• There is a TM for a language which halt every string.
• Halt on every string
• Accept or reject
• Recursively Enumerable Language
• There is a TM for a language which accepted a string otherwise not.
• Accept every string or not Halt( loop forever)
 Recursive and Recursively Enumerable Languages … Turing Computable

Total and Partial Functions

– A function f : N → N is total when f(n) is defined for every n
– Example: f(n) = 2n

Turing Computable … Turing Computable …

Computable Languages
Total and Partial Functions
– A language is computable if its characteristic function is computable (we can
– A function f : N → N is called partial when f(n) is defined for some n.
– Example: f(n) = logn use a Turing Machine to decide membership)
– For example {anbn , n≥0} is a computable language because there is a Turing
Machine that, given any string in W it decides whether the string belongs in the
language or not.
Partially Computable Languages
– If the characteristic function is partially computable, then the language is said to
End!!! be partially computable.
Thank You!!!
 Computational Complexity

– The field of complexity analysis is concerned with the study of the efficiency
of algorithms.

Chapter 7 Computational complexity – To compare the efficiency of algorithms, a measure of the degree of difficulty
of an algorithm called computational complexity was developed.
– Computational complexity indicates how much effort is needed to
execute an algorithm, or what its cost is.
– This cost can be expressed in terms of execution time (time efficiency, the
most common factor) or memory (space efficiency).

Computational Complexity … Big-O Notation

– The most commonly used notation for specifying asymptotic complexity, that is, for
– Time Complexity: Determine the approximate number of operations
estimating the rate of growth of complexity functions.
required to solve a problem of size n.
– Given two positive-valued functions f and g, consider the following definition:
– Space Complexity: Determine the approximate memory required to solve a
Definition:
problem of size n.
– The function f(n) is O(g(n)) if there exist positive numbers c and N such that f(n) ≤ c.g(n)
– The computational complexity of an algorithm is a measure of the cost (usually for all n ≥ N.
in execution time) incurred by applying the algorithm. – This definition states that g(n) is an upper bound on the value of f(n).
– The asymptotic complexity of an algorithm is an approximation of the – In other words, in the long run (for large n) f grows at most as fast as g.
computational complexity that holds for large amounts of input data. Example:
f(n) = n2 + 5n, f(n)=O(n2 )
 Big-O Notation … Big-O Notations …

Complexity Class Number of operations performed based on size of input data n

Name Big-O n=10 n=100 n=1000 n=10,000 n=100,000 n=1,000,000

Constant O(1) 1 1 1 1 1 1

Logarithmic O(log n) 3.32 6.64 9.97 13.3 16.6 19.93

Linear O(n) 10 100 1000 10,000 100,000 1,000,000

n log n O(n log n) 33.2 664 9970 133,000 1.66 * 106 1.99 * 107

Quadratic O(n2) 100 10,000 106 108 1010 1012

Cubic O(n3) 1000 106 109 1012 1015 1018

Exponential O(2n) 1024 1030 10301 103010 1030103 10301030

Class P vs class NP Class P vs class NP …

– A decision problem is in class P of polynomial-time problems if it can be solved by – Examples of tractable problems:

a deterministic Turing machine in a polynomial time as function of the size of the 1. determining whether an item is in a list of n elements.
2. determining whether a given positive integer n is prime.
input string,
NP class of decision problems
• there is a deterministic Turing machine T that solves the problem and halts in a
– A decision problem is in class NP of nondeterministic polynomial-time problems if it can
final state after no more than p(n) transitions when the input to T is a string of be solved by a nondeterministic Turing machine in a polynomial time as function of the
length n. size of the input string,
– Problems in P are called tractable while problems not in P are called intractable. – i.e., there is a nondeterministic Turing machine T that solves the problem and halts in
a final state after no more than p(n) transitions when the input to T is a string of
length n.
 Polynomial Time Reduction and NP-Complete Problems Polynomial Time Reduction and NP-…

– Input to a particular problem an instance of that problem. – P: (Decision) problems solvable by deterministic algorithms in polynomial time.
– Suppose that we have a procedure that transforms an y instance α of A in to – NP: (Decision) problems solved by non-deterministic algorithms in polynomial
some instance β of B. time.
– Given an instance α of Problem A, use a polynomial time reduction algorithm to – A group of(decision) problems have an additional important property.
transform it to an instance β of problem B. If any of them can be solved in polynomial time, then they all can!
– Run the polynomial-time decision algorithm for B on the instance β. – These problems are called NP-Complete problems.

Cook’s Theorem
• Theorem 10.1
• Satisfiability is NP-complete.
• Proof
• Satisfiability belongs to NP because a satisfying truth assignment serves as a
certificate for any yes-instance, which of course cab be checked in polynomial
time.
• We will now construct a collection Z(x) of clauses over some set V(x) of
Boolean variables for each x ϵ X, such that Z(x) is satisfiable if and only if x ϵ
Y.