FORMAL LANGUAGES AND AUTOMATA THEORY

Module 5

Rijin IK

Assistant Professor
Department of Computer Science and Engineering
Vimal Jyothi Engineering College
Chemperi

November 15, 2023

Rijin IK (VJEC) CST 301 M5 November 15, 2023 1 / 89

Outline
1 Course Outcomes
2 Context Sensitive Languages
3 Linear Bounded Automata (LBA)
4 Turing Machine (TM)
Turing Machines as Language Accepters
Turing Machine as Transducers
5 Robustness of the standard TM model
6 Multitape Turing Machines
7 Nondeterministic Turing machines
8 Universal Turing Machines
9 Halting Problem of TM
10 Recursive and Recursive Enumerable Language
11 Chomsky classification of formal languages
Rijin IK (VJEC) CST 301 M5 November 15, 2023 2 / 89
Course Outcomes

After the completion of the course the student will be able to

1 Classify a given formal language into Regular, Context-Free, Context
Sensitive, Recursive or Recursively Enumerable. [Cognitive knowledge
level: Understand]
2 Explain a formal representation of a given regular language as a finite
state automaton, regular grammar, regular expression and
Myhill-Nerode relation. [Cognitive knowledge level: Understand]
3 Design a Pushdown Automaton and a Context-Free Grammar for a
given context-free language. [Cognitive knowledge level : Apply]
4 Design Turing machines as language acceptors or transducers.
[Cognitive knowledge level: Apply]
5 Explain the notion of decidability. [Cognitive knowledge level:
Understand]

Rijin IK (VJEC) CST 301 M5 November 15, 2023 3 / 89

Context Sensitive Grammar
Context Sensitive Languages
A context sensitive grammar (CSG) is an unrestricted grammar in
which all the productions are of the form
α→β
Where α, β ∈ (V ∪ T )+ and |α| ≤ |β|.
Where α, β are strings of terminals and non- terminals.
A context sensitive grammar can never generate a language
containing the empty string ϵ; x → ϵ not allowed.
A context sensitive grammar (CSG) can be converted into normal
form where all productions are of the form,
αAβ → αγβ where γ ̸= ϵ
During derivation non-terminal A will be replaced by γ only when it is
present in context of α and β
This definition shows clearly one aspect of this type of grammar; it is
noncontracting, in the sense that the length of successive sentential
forms can never decrease. CST 301 M5
Rijin IK (VJEC) November 15, 2023 4 / 89
Context Sensitive Languages

Formal definition of Context Sensitive Grammar

A context sensitive grammar G = (V , T , P, S) , where
V is a set of nonterminal symbols
T is a set of terminal symbols
S is the start symbol, and
P is a set of production rules, of the form αAβ → αγβ where A in V,
α, β ∈ (V ∪ T )∗ and γ ∈ (V ∪ T )+
∗ Theproduction S → ϵ is also allowed if S is the start symbol and it does
not appear on the right side of any production.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 5 / 89

Context Sensitive Languages

Context Sensitive Language

A language L is said to be context-sensitive if there exists a
context-sensitive grammar G, such that L = L(G ).
If G is a Context Sensitive Grammar then,
+
L(G ) = {w |(w ∈ Σ∗ ) ∧ (S =⇒ w )}
G

Rijin IK (VJEC) CST 301 M5 November 15, 2023 6 / 89

Context Sensitive Languages
Example. The following grammar(G) is context-sensitive
S → aTb|ab
aT → aaTb|ac

L(G ) = {ab} ∪ {an cb n |n > 0}

This language is also a context-free.

For example, Context free grammar(G 1) for this.
S → aTb|ab
T → aTb|c

Any context-free language is context sensitive

Not all context-sensitive languages are context-free.
Rijin IK (VJEC) CST 301 M5 November 15, 2023 7 / 89
Context Sensitive Languages

Closure properties
Context Sensitive Languages are closed under
1 Union
2 Intersection
3 Complement
4 Concatenation
5 Kleene closure
6 Reversal

Rijin IK (VJEC) CST 301 M5 November 15, 2023 8 / 89

Linear Bounded Automata (LBA)

Linear Bounded Automata is a single tape Turing Machine with two

special tape symbols call them left marker ⊢ and right marker ⊣.
It is allowed to read/write between these markers.
It should not replace the marker symbols by any other symbol.
It should not write on cells beyond the marker symbols.
Context sensitive languages are recognised using linear bounded
automata (LBA)
Here input tape is restricted in size. A linear function is used for
restricting the length of the input tape.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 9 / 89

Linear Bounded Automata (LBA)

Figure: Linear Bounded Automata (LBA)

Rijin IK (VJEC) CST 301 M5 November 15, 2023 10 / 89

Linear Bounded Automata (LBA)

Read/ write tape:

Read/ write head can move to right/left, one cell at a time.
The tape is initally assumed to be store the input delimited by ⊣ and
the blank symbol B, which is followed by blank symbol (⊣ )in all the
cells until the cell containing ⊢
If the input size is n, the number of cells in the tape which the
machine can use is bounded by a linear function of n.
The machine can scan the input in right/left directions any number of
rounds
Finite Control:
Control unit with a finite set of states which can remember finite
amount of information about the processed input.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 11 / 89

Linear Bounded Automata (LBA)

Formal definition
Formally Linear Bounded Automata is a non-deterministic Turing
Machine, M = (Q, Σ, Γ, δ, B, ⊢, ⊣, q0 , t, r )
Q is set of all states
Σ is set of all terminals
Γ is set of all tape alphabets Σ ⊂ Γ
δ is set of transitions (Q × Γ) → (Q × Γ × {R, L})
B is blank symbol (B ∈ Γ)
⊢ is left marker and ⊣ is right marker (⊢ and ⊣ ∈ Γ)
q0 is the initial state (q0 ∈ Q)
t is accept state (t ∈ Q)
r is reject state (r ∈ Q)

Rijin IK (VJEC) CST 301 M5 November 15, 2023 12 / 89

Linear Bounded Automata (LBA)
Example:Consider an LBA defined as,

Q = {q0, q1, q2, q3, q4}

Σ = {a, b, c}
Γ = {a, b, c, B}
q0 = q0
δ(q0, ⊢) = (q1, ⊢, R) δ(q1, ⊢) = (q1, ⊢, R) δ(q1, B) = (q1, B, R)
δ(q1, a) = (q2, B, R) δ(q2, a) = (q2, a, R) δ(q2, B) = (q2, B, R)
δ(q2, b) = (q3, B, R) δ(q3, b) = (q3, b, R) δ(q3, B) = (q3, B, R)
δ(q3, c) = (q4, B, L) δ(q4, c) = (q4, c, L) δ(q4, b) = (q4, b, L)
δ(q4, a) = (q4, a, L) δ(q4, B) = (q4, B, L) δ(q4, ⊣) = (q1, ⊢, R)

In the above, the transition δ(q3, b) = (q3, b, R) means,LBA on state q3,

head points to symbol b, remains in state q3, replaces b with b and head
turns towards right by one cell
Rijin IK (VJEC) CST 301 M5 November 15, 2023 13 / 89
Linear Bounded Automata (LBA)
Example:1A.Design Linear Bounded Automata for the language
L = {an b n c n |n ≥ 1}.
Q = {q0, q1, q2, q3, t, r }
Σ = {a, b, c}
Γ = {a, b, c, X , Y , Z }
q0 = q0
t = t(accept state) , r = r (reject state)
δ(q0 , ⊢) = (q0 , ⊢, R) δ(q0 , a) = (q1 , X , R) δ(q1 , a) = (q1 , a, R)
δ(q1 , b) = (q2 , Y , R) δ(q2 , b) = (q2 , b, R) δ(q2 , c) = (q3 , Z , L)
δ(q3 , b) = (q3 , b, L) δ(q3 , Y ) = (q|3 , Y , L) δ(q3 , a) = (q3 , a, L)
δ(q3 , X ) = (q0 , X , R) δ(q2 , Z ) = (q2 , Z , R) δ(q3 , Z ) = (q3 , Z , L)
δ(q1 , Y ) = (q1 , Y , R) δ(q0 , Y ) = (q0 , Y , R) δ(q0 , Z ) = (q0 , Z , R)
δ(q0 , B) = (t, B, L) δ(q0 , b) = (r , b, R) δ(q0 , c) = (r , c, R)
δ(q1 , c) = (r , c, R) δ(q2 , B) = (r , B, L)
Rijin IK (VJEC) CST 301 M5 November 15, 2023 14 / 89
Turing Machine (TM)

Figure: Chomsky Hierarchy

From the diagram, unrestricted grammars produce unrestricted languages

(Type-0 languages). These Type-0 languages are recognised using Turing
machines

Rijin IK (VJEC) CST 301 M5 November 15, 2023 15 / 89

Turing Machine (TM)

Example of unrestricted grammars:

S → ACaB
CaBc → aaC
CB → DB
aD → Da
aEC → Ea
AE → ϵ

Here LHS and RHS can contain any set of terminals and non terminals.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 16 / 89

Turing Machine (TM)

Figure: Turing Machine (TM)

The basic model has a finite control, an input tape that is divided into
cells, and a tape head that scans one cell of the tape at a time.
Rijin IK (VJEC) CST 301 M5 November 15, 2023 17 / 89
Turing Machine (TM)
Tape
The tape has a leftmost cell but is infinite to the right.
Each cell of the tape may hold exactly one of a finite number of tape
symbols.
Initially, the n leftmost cells, for some finite n ≥ 0, hold the input,
which is a string of symbols chosen from a subset of the tape symbols
called the input symbols.
The remaining infinity of cells each hold the blank, which is a special
tape symbol that is not an input symbol.
Finite control
At a particular instant, finite control is in a state. States are divided
into,
Start state (q0),
Halt state (h),
Intermediate states (q1, q2, .....).
Tape head
Tape head points to one of the tape cells. It communicates between
finite control and tape. It can move left , right or remain stationary.
Rijin IK (VJEC) CST 301 M5 November 15, 2023 18 / 89
Turing Machine (TM)

Moves of Turing Machine

In one move the Turing machine, depending upon the symbol scanned by
the tape head and the state of the finite control,
1 changes state
2 prints a symbol on the tape cell scanned, replacing what was written
there, and
3 moves its head left or right one cell.
The difference between a Turing machine and a two-way finite automaton
lies in the former’s ability to change symbols on its tape.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 19 / 89

Turing Machine (TM)
Formal definition-Turing Machine (TM)
Turing Machine M is denoted by the 9-tuple:
M = (Q, Σ, Γ, δ, B, ⊢, q0 , t, r )
Q is set of all states
Σ is a finite set (the input alphabet)
Γ is set of all tape alphabets Σ ⊂ Γ
δ is set of transitions (Q × Γ) → (Q × Γ × {R, L})
B is blank symbol (B ∈ Γ)
⊢ is left marker (⊢∈ Γ)
q0 is the initial state (q0 ∈ Q)
t is accept state (t ∈ Q)
r is reject state (r ∈ Q)
Intuitively, δ(p, a) = (q, b, d) means, ”When in state p scanning symbol a,
write b on that tape cell, move the head in direction d(L/R), and enter
state q. The symbols Land R stand for left and right, respectively.
Rijin IK (VJEC) CST 301 M5 November 15, 2023 20 / 89
Turing Machine (TM)

Instantaneous description (ID):

Instantaneous description of the Turing machine M is denoted by
α1 qα2
Here q, the current state of M, is in Q; α1 α2 is the string in Γ∗ that is
the contents of the tape up to the rightmost nonblank symbol or the
symbol to the left of the head, whichever is rightmost. (Observe that
the blank B may occur in α1 α2 .)
The tape head is assumed to be scanning the leftmost symbol of α2 ,
or if α2 = ϵ, the head is scanning a blank.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 21 / 89

Turing Machine (TM)

Define a move in TM
Let X1 X2 . . . Xi−1 qXi . . . Xn be an ID.
The left move is: if δ(q, Xi ) = (p, Y , L) ,if i > 1 then
X1 X2 . . . Xi−1 qXi . . . Xn ⊢ X1 X2 . . . Xi−2 pXi−1 YXi+1 . . . Xn .
The right move is: if δ(q, Xi ) = (p, Y , R) ,if i > 1 then
X1 X2 . . . Xi−1 qXi . . . Xn ⊢ X1 X2 . . . Xi−1 YpXi+1 . . . Xn .

Rijin IK (VJEC) CST 301 M5 November 15, 2023 22 / 89

Turing Machine (TM)

Language accepted by a turing machine

The language of a Turing machine M, denoted L(M), is the set of all
strings that M accepts:

L(M) = {w ∈ Σ∗ |M accepts w }

TM either accepts or rejects a string. That is, TM acts as a language

acceptor.
A TM is said to accept an input string w if :
∗
(q0 , ⊢ wB W , 0) =
⇒ (t, y , n), y ∈ Γw , n ∈ N
A TM is said to reject an input string w if :
∗
(q0 , ⊢ wB W , 0) =
⇒ (r , y , n), y ∈ Γw , n ∈ N

Rijin IK (VJEC) CST 301 M5 November 15, 2023 23 / 89

Turing Machines as Language Accepters

Turing Machines as Language Accepters

Turing machines can be viewed as accepters in the following sense.
String w is written on the tape, with blanks filling out the unused
portions.
The machine is started in the initial state q0 with the read-write head
positioned on the leftmost symbol of w.
If, after a sequence of moves, the Turing machine enters a final state
and halts, then w is considered to be accepted.
Definition
Let M = (Q, Σ, Γ, δ, B, ⊢, q0 , t, r ) be a Turing machine. Then the
language accepted by M is

L(M) = {w ∈ Σ+ : q0 w ⊢∗ x1 qf x2 for some qf ∈ t, x1 , x2 ∈ Γ∗ }.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 24 / 89

Turing Machine (TM)

Q:Design a TM to accept the language L = {0n 1n |n ≥ 1}.

Initially, the tape of M contains 0n 1n followed by infinity of blanks.
Repeatedly, M replaces the leftmost 0 by X, moves right to the
leftmost 1, replacing it by Y, moves left to find the rightmost X, then
moves one cell right to the leftmost 0 and repeats the cycle.
If, however, when searching for a 1, M finds a blank instead, , then M
halts without accepting.
If, after changing a 1 to a Y, M finds no more 0’s, then M checks
that no more 1’s remain, accepting if there are none

Rijin IK (VJEC) CST 301 M5 November 15, 2023 25 / 89

Turing Machine (TM)

Figure: Transition table

Rijin IK (VJEC) CST 301 M5 November 15, 2023 26 / 89

Turing Machine (TM)

Figure: Transition diagram

Rijin IK (VJEC) CST 301 M5 November 15, 2023 27 / 89

Turing Machine (TM)

String Verfication by Turing Machine

Figure: A computation of M

Rijin IK (VJEC) CST 301 M5 November 15, 2023 28 / 89

Turing Machine (TM)

TM as transducers
A TM can function as a transducer. That is, a string is given as input to
TM, it produces some output.
Input to the computation is a set of symbols on the tape.
At the end of computation, whatever remains on the tape is the
output.
A TM can be viewed as a transducer for the implementation of function f
defined as,y = f (x) for strings x, y ∈ Σ∗
A function, f is said to be computable or Turing computable, if there
exists a TM M such that q0 x ⊢∗ qf f (x)
where x is in the domain of f.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 29 / 89

Turing Machine (TM)

TM is an abstract model of our modern computer system

We can see that all common mathematical functions are turing
computable.
This means, basic operations such as addition, subtraction,
multiplication, division can be performed on it.
This means that a TM is an abstract model of our modern computer
system.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 30 / 89

Turing Machine (TM)

Unary Representation
In computing functions using a Turing machine (TM) where inputs
are represented using unary notation, a non-negative integer n is
indeed represented as a sequence of 1s or 0s.
For example:
If n = 2, it is represented as ”11” or ”00”
If n = 4, it is represented as ”1111” or ”0000”

Rijin IK (VJEC) CST 301 M5 November 15, 2023 31 / 89

Turing Machine (TM)

Q:Design a Turing machine that computes the following function,

f (m, n) = m + n.

A positive integer on a Turing tape can be represented by an equal

number of 0s.For example,
integer 5 is represented as 00000, and
integer 8 is represented as 00000000.
In the tape two integers are separated using the symbol, $.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 32 / 89

Turing Machine (TM)
Let us assume that m=3 and n=5.Then the input tape of the Turing
machine will be,

After addition, tape will contain the results of addition as shown below

TM for addition works as follows:

Head reads the 0s of the first number, m and reaches the separator
symbol, $. Symbol, $ is replaced with 0 and head moves towards right
It continues to move towards right till the second number, n is passed
over and B is reached.
At B, it turns left and replaces rightmost 0 with B.
Now the tape contains the sum of m and n, and TM halts.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 33 / 89

Turing Machine (TM)

Conversion of the separator symbol, $ to 0, helps TM to make the single

number on the tape. The conversion of symbol, 0 to B removes the
additional 0 that was generated from the conversion of $ to 0.
Transition table for the TM is shown below:

Figure: Transition table

Rijin IK (VJEC) CST 301 M5 November 15, 2023 34 / 89

Turing Machine (TM)

The Transition table is describes as follows:

1 Initially, TM is in state q0 . Head reads the first symbol, 0 and moves
towards right without changing the state.
2 Head continues to move towards right till the separator symbol,$ is
reached. It replaces $ with 0, changes state to q1, and continues to
move towards right.
3 Head continues to move towards right and passes over the second
number to reach B.
4 On reaching the B, head changes state to q2 and turns left.
5 In state q2 , it replaces symbol 0 with B, and changes state to qf .
6 In the final state, TM halts to indicate the completion of the process.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 35 / 89

Turing Machine (TM)
Addition of 3 and 5 is shown below

Rijin IK (VJEC) CST 301 M5 November 15, 2023 36 / 89

Turing Machine (TM)

Rijin IK (VJEC) CST 301 M5 November 15, 2023 37 / 89

Turing Machine (TM)

Rijin IK (VJEC) CST 301 M5 November 15, 2023 38 / 89

Robustness of the standard TM model

Robustness of the standard TM model

A computational model is robust if the class of languages it accepts
does not change under variants.
The robustness of Turing Machines is by far greater than the
robustness of DFAs and PDAs.
We introduce several variants on Turing machines and show that all
these variants have equal computational power.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 39 / 89

Robustness of the standard TM model

Types of Turing Machines

Two-way infinite tape
Multitape Turing machines
Multi-track Turing Machine
Nondeterministic Turing machines
Multidimensional Turing machines
Off-line Turing machines
Universal Turing Machines

Rijin IK (VJEC) CST 301 M5 November 15, 2023 40 / 89

Two-way infinite tape
Two-way infinite tape
The TM we discussed so far had a tape that was finite on the left end
and infinite on the right end.
Two way infinite TM is an extension to this such that tape is infinite
at both ends. This is shown below:

Figure: Two-way infinite tape

Thus on both sides of the tape, there is an infinite sequence of blank
symbols(B).
This model does not provide any additional computational capability.
Rijin IK (VJEC) CST 301 M5 November 15, 2023 41 / 89
Multitape Turing Machines

Multitape Turing Machines

A multitape TM consists of multiple tapes. Each tape has a separate
head

Figure: Multitape Turing Machines

Rijin IK (VJEC) CST 301 M5 November 15, 2023 42 / 89

Multitape Turing Machines

Multitape Turing Machines

There are n tapes, each divided into cells. The first tape holds the
input string. Initially, all the other tapes hold the blank symbol, B
A head has three possible options:
to move towards left (L),
to move towards right (R), or
to remain stationary (N).
All the heads are connected to a finite control. Finite control is in a
state at an instant.
1 Initially, finite control is in state q0 .
2 Head in the lowermost tape points to the cell containing leftmost
symbol of the input string.
3 All the cells in the upper tapes contain B symbol

Rijin IK (VJEC) CST 301 M5 November 15, 2023 43 / 89

Multitape Turing Machines

When a transition occurs,

1 Finite control may change its state.
2 Head reads the symbol from the current cell and writes a symbol on it.
3 Each head can move towards left (L), right (R) or stay stationary (N)
An example transition function for a 4 tape TM is given below:
δ(q1 , [a1 , a2 , a3 , a4 ]) = (q2 , [b1 , b2 , b3 , b4 ], [L, R, R, N])
Here the finite control is in state q1 , It reads the symbol a1 from the
uppermost tape, a2 from the next uppermost tape and so on.
After reading, finite control changes its state to q2 , and replaces the
symbol a1 by b1 , a2 by b2 , a3 by b3 , a4 by b4 .
After this head in the upper most tape turns left. Head in the 2nd
tape turns right. Head in the 3rd tape turns right. Headin the 4th
tape remains stationary.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 44 / 89

Multitape Turing Machines
Example:Consider the language L = {an b n n ≥ 1}.
In a normal TM, head has to move back and forth to match each pair
of symbols a and b. On a multitape TM, no such movements are
needed.
This is done by making a copy of the input string in another tape.
Let the input string be aaaabbbb.
Let the input srtring be in tape 1. Make a copy of this string in tape 2

Rijin IK (VJEC) CST 301 M5 November 15, 2023 45 / 89

Multitape Turing Machines

The head of tape 1 is positioned on the first a of the input string.

Head of tape 2 is positioned on the first b of the input string.
Now the heads advance on both tapes simultaneously towards right
and the string is accepted if there are equal number of a’s and b’s in
the string.
This will happen if the head on tape 1 encounters the first b and the
head on tape 2 encounters the first B(blank symbol) simultaneously.
∗ Every
multitape Turing Machines has an equivalent single tape Turing
machine.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 46 / 89

Multi-track Turing Machine
Multi-track Turing Machine
Multi-track Turing machines, a specific type of Multi-tape Turing
machine, contain multiple tracks but just one tape head reads and
writes on all tracks.
Here, a single tape head reads n symbols from n tracks at one step.
It accepts recursively enumerable languages like a normal single-track
single-tape Turing Machine accepts.

Figure: Multi-track Turing Machines

Rijin IK (VJEC) CST 301 M5 November 15, 2023 47 / 89

Multitape Turing Machines

Theorem:
Every language accepted by a k-tape TM M1 is also accepted by a single
tape TM M2
Proof by construction
Let L be accepted by M1 ,a TM with k-tapes. We can construct M2,
a one-tape TM with 2k tracks, two tracks for each of M1’s tapes.
One track records the contents of the corresponding tape of M1
and the other is blank, except for a marker in the cell that holds the
symbol scanned by the corresponding head of M1.
The finite control of M2 stores the state of M1 ,along with a count of
the number of head markers to the right of M2’s tape head.H

Rijin IK (VJEC) CST 301 M5 November 15, 2023 48 / 89

Multitape Turing Machines

Figure: Simulation of multi tapes by one

Rijin IK (VJEC) CST 301 M5 November 15, 2023 49 / 89

Multitape Turing Machines

To simulate one move of the k-tape TM:

Each move of M1 is simulated by a sweep from left to right and then
from right to left by the tape head of M2.
Initially, M2’s head is at the leftmost cell containing a head marker.
To simulate a move of M1, M2 sweeps right, visiting each of the celis
with head markers andrecording the symbol scanned by each head of
M1.
When M2 crosses a head marker, it mustupdate the count of head
markers to its right.
When no more head markers are to the right, M2 has seen the
symbols scanned by each of M1’s heads, so M2 has enough
information todetermine the move of M1.
Now M2 makes a pass left, until it reaches the leftmost headmarker.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 50 / 89

Multitape Turing Machines

The count of markers to the right enables M2 to tell when it has gone
for enough.
As M2 passes each head marker on the leftward pass, it updates the
tape symbol of M1 “scanned”by that head marker, and it moves the
head marker one symbol left or right to simulate themove of M1.
Finally, M2 changes the state of M1 recorded in M2’s control to
complete thesimulation of one move of M1.
If the new state of M1 is accepting, then M2 accepts.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 51 / 89

Nondeterministic Turing machines

Nondeterministic Turing machines

A nondeterministic Turing machine is a device with a finite control
and a single, one-way infinite tape.
For a given state and tape symbol scanned by the tape head, the
machine has a finite number of choices for the next move.
Each choice consists of a new state, a tape symbol to print, and a
direction of head motion.
Note that the nondeterministic TM is not permitted to make a move
in which the next state is selected from one choice, and the symbol
printed and/or direction of head motion are selected from other
choices.
The nondeterministic TM accepts its input if any sequence of choices
of moves leads to an accepting state.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 52 / 89

Nondeterministic Turing machines

Formal definition- Nondeterministic Turing Machine

Nondeterministic Turing Machine M is denoted by the 9-tuple:
M = (Q, Σ, Γ, δ, B, ⊢, q0 , t, r )
Q is set of all states
Σ is a finite set (the input alphabet)
Γ is set of all tape alphabets Σ ⊂ Γ
δ is set of transitions (Q × Γ) → 2(Q×Γ×{R,L})
B is blank symbol (B ∈ Γ)
⊢ is left end marker (⊢∈ Γ)
q0 is the initial state (q0 ∈ Q)
t is accept state (t ∈ Q)
r is reject state (r ∈ Q)
t, r ∈ Q are sink states

Rijin IK (VJEC) CST 301 M5 November 15, 2023 53 / 89

Nondeterministic Turing machines

Figure: Nondeterministic Turing machine

Rijin IK (VJEC) CST 301 M5 November 15, 2023 54 / 89

Nondeterministic Turing machines
Every nondeterministic Turing Machine has an equivalent deterministic
Turing machine
Theorem
If L is accepted by a single-tape NDTM M1 , then L is accepted by
some DTM M2 .
Let d be the maximum number of nondeterministic choices M1 can
make at any given move.
M2 will systematically try all nondeterministic possibilities.
M2 will have three tapes.
M2 uses 3 tapes:
Tape 1 holds the input,It’s reused many times.
Tape 2 contains a sequence of digits from 1 to d, generated
systematically. Each sequence dictates a sequence of choices. eg:
. [1 ],[ 2 ], [3 ], ... , [d] ,
. [1,1] , [1, 2] , [2, 1] , ... , [d, d] ,
. [1,1,1] , [1,1,2 ], [1, 2, 1] , ... , [d, d, d] ,
Tape 3 contains a scratch copy of the input.
Rijin IK (VJEC) CST 301 M5 November 15, 2023 55 / 89
Nondeterministic Turing machines

Figure: The computational tree of a non-deterministic Turing machine

Rijin IK (VJEC) CST 301 M5 November 15, 2023 56 / 89

Nondeterministic Turing machines

Figure: Deterministic TM simulating Nondeterministic Turing machine

Rijin IK (VJEC) CST 301 M5 November 15, 2023 57 / 89

Nondeterministic Turing machines

How M2 works :
1 Generate the next sequence on tape 2
2 Copy tape 1 (input) to tape 3 (scratch copy)
3 Simulate M1 on tape 3, making choices according to the sequence on
tape 2.
4 If M1 accepts, then accept;
5 Otherwise, go to step 1.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 58 / 89

Universal Turing Machines

Universal Turing Machines

A Turing machine (TM) U is called a universal Turing machine
(UTM) if it is able to simulate all other TMs:
for all TMs M and inputs w, U(< M, w >) = M(w ).
A Universal turing Machine (UTM) takes two arguments
1 The description of a normal TM
2 The description of input string w and checks whether w is recognised
The description of TM and input string w is the encoded string over a
finite alphabet set.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 59 / 89

Universal Turing Machines

Unary Encoding of the elements in a turing machine

Rijin IK (VJEC) CST 301 M5 November 15, 2023 60 / 89

Universal Turing Machines

Total coding(Binary Encoding) of the TM is

Rijin IK (VJEC) CST 301 M5 November 15, 2023 61 / 89

Universal Turing Machines

Encoding of an input string

Consider Σ = {a, b}
Using unary encoding on the alphabet set {1}
a can be represented as 1 and
b can be represented as 11.
and each successive character representations are separated with a 0
Let w = abab be the string to be checked on the Turing machine, M.
The input to UTM will be
ŵ = 101101011
UTM uses the binary code of the turing machine, M on string abab
and will check if abab is recognised by M. If true UTM, will halt to
say yes and if false UTM will stop to say no.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 62 / 89

Universal Turing Machines

Figure: Universal Turing Machines

Figure: Input to Universal Turing Machines

Rijin IK (VJEC) CST 301 M5 November 15, 2023 63 / 89

Universal Turing Machines

U takes as input the binary encoding M̂ of turing Machine M and the

binary encoding ŵ of an input w to M, seperated by a #
(ΣU = {0, 1, #})
Given the input string M̂#ŵ :
Checks whether M̂ is a valid encoding of a Turing Machine and ŵ is a
valid encoding of a string. Rejects if encoding is not valid
Run M on w and:
a Accepts if M accepts w.
b Rejects if M rejects w.
c Loops if M loop on w

Rijin IK (VJEC) CST 301 M5 November 15, 2023 64 / 89

Universal Turing Machines

Figure: Structure of a UTM

Tape 1 storing the encoding M̂ of the TM M

Tape 2 storing the encoding ŵ of the i/p w
Tape 3 storing the Current state of M and the current position of
tape head in M
Rijin IK (VJEC) CST 301 M5 November 15, 2023 65 / 89
Universal Turing Machines

U repeatedly performs the following, to simulate a single step of M on

w:
1 Scan tape 3 and find M’s current state and the current position of the
tape head.
2 Scan tape 2 and find the next input symbol of M
3 Scan tape 1 and find the transition to apply in the current
configuration of M.
4 Apply the transition in the current configuration of M and accordingly ,
update tape2 and 3
5 If M enters tM rM then enter tU rU

Rijin IK (VJEC) CST 301 M5 November 15, 2023 66 / 89

Halting Problem of TM

Halting Problem of TM
In computability theory, the halting problem is the problem of
determining, from a description of an arbitrary computer program and
an input, whether the program will finish running, or continue to run
forever.
Alan Turing proved in 1936 that a general algorithm to solve the
halting problem for all possible program-input pairs cannot exist.
Halting problem is undecidable.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 67 / 89

Halting Problem of TM

Problem
Given the description of a Turing machine M and an input w, does M,
when started in the initial configuration q0 w , perform a computation
that eventually halts?
Input: A Turing machine and an input string w.
Problem: Does the Turing machine finish computing of the string w
in a finite number of steps? The answer must be either yes or no

Rijin IK (VJEC) CST 301 M5 November 15, 2023 68 / 89

Halting Problem of TM
Proof:
At first, we will assume that such a Turing machine exists to solve
this problem and then we will show it is contradicting itself.
We will call this Turing machine as a Halting machine that produces
a ‘yes’ or ‘no’ in a finite amount of time.
If the halting machine finishes in a finite amount of time, the output
comes as ‘yes’, otherwise as ‘no’.
The following is the block diagram of a Halting machine.

Figure: Halting machine

Rijin IK (VJEC) CST 301 M5 November 15, 2023 69 / 89

Halting Problem of TM
Proof cont..
Now we will design an inverted halting machine (HM)’ as
If H returns YES, then loop forever.
If H returns NO, then halt.
The following is the block diagram of an ‘Inverted halting machine’

Figure: Inverted halting machine (HM)’

Rijin IK (VJEC) CST 301 M5 November 15, 2023 70 / 89

Halting Problem of TM
Proof cont..
Further, a machine HM2 which input itself is constructed as follows
If HM2 halts on input, loop forever.
Else, halt.
Here, we have got a contradiction. Hence, the halting problem is
undecidable.

Figure: Inverted halting machine (HM)’

Rijin IK (VJEC) CST 301 M5 November 15, 2023 71 / 89

Recursive and Recursive Enumerable Language

Recursive set
A set is called recursively enumerable if there’s a step-by-step
method, like using a specific rule or algorithm, that can eventually list
out all the numbers in that set.
However, this method might take a while, and it can’t quickly tell you
whether a specific number is in the set or not. So, it’s a way to
gradually list the numbers but not a straightforward way to decide if a
particular number belongs to the set.
The union and intersection of two recursively enumerable sets are also
recursively enumerable.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 72 / 89

Recursive and Recursive Enumerable Language

Recursive set
A set S of integers is said to be recursive if there is a total recursive
function f (x) such that f (x) = 1 for x in S and f (x) = 0 for x not in
S.
Any recursive set is also recursively enumerable.
Finite sets, sets with finite complements, the odd numbers, and the
prime numbers are all examples of recursive sets.
The union and intersection of two recursive sets are themselves
recursive, as is the complement of a recursive set.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 73 / 89

Recursive and Recursive Enumerable Language
Recursively enumerable languages
A language is said to be recursively enumerable if there exists a
Turing Machine that accepts every string of the language and rejects
the string that are not in the language and it may cause TM to enter
into an infinite loop.
Turing machines for recursively enumerable languages may not always
halt, and continue to run endlessly for strings that are not a part of
that language. Recursively enumerable languages are also called
Turing recognizing languages.

Figure: The output of a TM running over a recursively enumerable language

Rijin IK (VJEC) CST 301 M5 November 15, 2023 74 / 89

Recursive and Recursive Enumerable Language

Recursive Language
A language is said to be recursive if there exists a Turing Machine
that accepts every string of the language and rejects the string that
are not in the language.
A TM of a recursive language never goes into a loop and always halts
in every case. Recursive languages are hence also called Turing
decidable languages.

Figure: The output of a TM running over a recursive language

Rijin IK (VJEC) CST 301 M5 November 15, 2023 75 / 89

Recursive and Recursive Enumerable Language
All the Recursive Languages are Recursively Enumerable.
Every Recursively Enumerable Languages are not Recursive.
Recursive Languages are subset of Recursively Enumerable Languages.

Figure: Recursive and Recursive Enumerable Language

Rijin IK (VJEC) CST 301 M5 November 15, 2023 76 / 89

Recursive and Recursive Enumerable Language

Closure properties of recursive languages

Union
Intersection
Complement
Kleene’s closure

Rijin IK (VJEC) CST 301 M5 November 15, 2023 77 / 89

Recursive and Recursive Enumerable Language
Union:
The union of two recursive languages L1 and L2 is also a recursive
language.
That is, if the TM accepts any one of the languages, it also accepts
L1 ∪ L2

Figure: The union of two recursive languages

Rijin IK (VJEC) CST 301 M5 November 15, 2023 78 / 89

Recursive and Recursive Enumerable Language
Intersection:
The intersection of two recursive languages L1 and L2 is also a
recursive language.
That is, if the TM accepts both languages, it also accepts L1 ∩ L2

Figure: The intersection of two recursive languages

Rijin IK (VJEC) CST 301 M5 November 15, 2023 79 / 89

Recursive and Recursive Enumerable Language

Complement:
The complement of a recursive language L1 is also a recursive
language.
That is, if the TM accepts L1 it rejects it as a final output.

Figure: The complement of a recursive language

Rijin IK (VJEC) CST 301 M5 November 15, 2023 80 / 89

Recursive and Recursive Enumerable Language

If L is recursive, then so is L̄.

LetL be decided by a deterministic M.
Swap the ”accept” state and the ”reject” state of M.
The new machine decides L̄

Rijin IK (VJEC) CST 301 M5 November 15, 2023 81 / 89

Recursive and Recursive Enumerable Language

Kleene’s closure:
The Kleene’s closure of a recursive language L1 is also recursive
That is, if L1 = {o i 1j } is recursive, then L∗1 = {o i 1j }∗ is also
recursive.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 82 / 89

Recursive and Recursive Enumerable Language

Closure properties of recursive Enumerable Language

Recursive Enumerable Languages are closed under
Union
Intersection
Concatenation
Kleene’s closure
∗ The complement of a recursively enumerable language is not necessarily
recursively enumerable. There exist recursively enumerable languages L
such that L̄(complement of L) is not recursively enumerable.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 83 / 89

Recursive and Recursive Enumerable Language

Theorem:
If L is recursively enumerated and L̄ is recursively enumerable then L is
recursive. Proof
Let M and M’ be TMs that accept L and L̄, respectively.
Run M and M’ simultaneously.
For any word x, it must be accepted by one of M or M’
So, either M or M’ will halt and accept
If M halts and accepts then M’ halt and reject
If M halts and reject then M’ halt and accepts
The TM that runs M and M’ simultaneously always halts and accepts or
rejects so it decides L and L is recursive.

Rijin IK (VJEC) CST 301 M5 November 15, 2023 84 / 89

Chomsky classification of formal languages

Rijin IK (VJEC) CST 301 M5 November 15, 2023 85 / 89

Chomsky classification of formal languages
Type 0
All formal grammars fall under type-0 grammars.
The Turing machine can detect languages with type 0 grammar.
The Recursively Enumerable languages are another name for these
languages.
Type 0 Language Production in the form of α → β where
α , β ∈ (V + T )∗
V = Variables
T = Terminals.
In type 0 there must be at least one variable on the Left side of
production.
For example:
S → ACaB
Bc → acB
CB → DB
aD → Db
Rijin IK (VJEC) CST 301 M5 November 15, 2023 86 / 89
Chomsky classification of formal languages
Type 1
It is also known as Context Sensitive Language.
It is recognized by the Linear Bound Automata.
According to the following rules, context-sensitive grammar:
It should be Type 0
Multiple symbols may be present on the left side of the production
rules for the context-sensitive grammar.
The number of symbols on the left side cannot be more than the
number of symbols on the right side.
In type 1, Production is in the form of αAβ → αγβ where A in V, α,
β ∈ (V ∪ T )∗ and γ ∈ (V ∪ T )+
The rule of the form A → ϵ is not allowed unless A is a start symbol.
For Example:
S → AB
AB → abc
B→b
Rijin IK (VJEC) CST 301 M5 November 15, 2023 87 / 89
Chomsky classification of formal languages

Type 2
It should be Type 1
It is Context-Free Grammar (CFG)
It is recognized by a Pushdown automata
The left-hand side of production can have only one variable and there
is no restriction on right side
Production is of the form A → α,
where A is a variable and α is a string of symbols from (V ∪ T )∗ .
For Example:

S → AB
A→a
B→b

Rijin IK (VJEC) CST 301 M5 November 15, 2023 88 / 89

Chomsky classification of formal languages
Type 3
It should be Type 2
It is a Regular Grammar
It can be accepted by a finite-state automaton.
It is the most restricted form of grammar.
It can be described using regular expressions.
These languages can be modelled by NFA or DFA.
Production is of the form
V → VT /T (left − linear regular grammar )
(or )
V → TV /T (right − linear regular grammar )
For Example:
S → aS
S → bS
S →ab
Rijin IK (VJEC) CST 301 M5 November 15, 2023 89 / 89