THEORY OF AUTOMATA

Mukesh Kumar Tripathi

Assistant Professor
Department of Information Technology
Vasavi College of Engineering
 Recursive and Recursively Enumerable Languages
• When a Turing machine reaches a final state, it ‘halts’.
• We can also say that a Turing machine M halts when M reaches a state q and a current
symbol a to be scanned so that δ(q, a) is undefined.
• There are TMs that never halt on some inputs in any one of these ways.
• So we make a distinction between the languages accepted by a TM that halts on all input
strings and a TM that never halts on some input strings.

TM Execution Possible Outcomes

• Accept: Running TM M on input w eventually leads to qaccept.
• Reject: Running TM M on input w eventually leads to qreject.
• No Decision: Running TM M on input w runs forever, never reaches qaccept or qreject.
Recursive and Recursively Enumerable Languages (contd.)
Recursively Enumerable Language: A language L is recursively enumerable (Turing-
recognizable) if there exists a TM M such that for all strings w:
• If w  L then M halts in a final state.
• If w  L then M may halt in a non-final state, or loop forever.

Recursive Language: A language L is recursive (Turing-decidable) if there exists a TM M

such that for all strings w:
• If w  L then M accepts w and halts.
• If w  L then M eventually halts, without reaching an accepting state.
• A problem/language is undecidable if it is not decidable.
• The set of all recursive languages is a subset of the set of all recursively enumerable
languages.
Recursive and Recursively Enumerable Languages (contd.)
• TM Block Diagrams:
• If L is a recursive language, then a TM M that accepts L and always halts can be
pictorially represented by a “chip” that has one input and two outputs.
yes
w M
no

• If L is a recursively enumerable language, then a TM M that accepts L can be pictorially

represented by a “chip” that has one output.

yes
w M
 Properties of Recursive and Recursively Enumerable Languages
Theorem: The recursive languages are closed with respect to union, i.e., if L1 and L2 are
recursive languages, then so is L1 L2
Properties of Recursive and Recursively Enumerable Languages (contd.)
Theorem: The recursively enumerable languages are closed with respect to union, i.e., if L1
and L2 are recursively enumerable languages, then so is L1 L2
Properties of Recursive and Recursively Enumerable Languages (contd.)
Theorem: The recursive languages are closed with respect to complementation, i.e., if L is a
recursive language, then so is
Properties of Recursive and Recursively Enumerable Languages (contd.)
Theorem: If L and are both recursively enumerable then L (and therefore ) is recursive.
 The Church-Turing Thesis
• The Church-Turing thesis states that any algorithmic procedure that can be carried out by a
human or a computer, can also be carried out by a Turing machine.
• Statements that are implied by the Church-Turing thesis:
• Any mechanical computation can be performed by a TM.
• There is some TM corresponding to every computable problem.
• The set of languages that can be decided by a TM is identical to the set of languages that
can be decided by any mathematical computing machine.
• If there is no TM that decides problem P, then there is no algorithm that solves problem P.
 Decidability vs. Undecidability
• There are two types of TMs (based on halting):
(Recursive)
TMs that always halt, no matter accepting or non-accepting  DECIDABLE
PROBLEMS
(Recursively enumerable)
TMs that are guaranteed to halt only on acceptance. If non-accepting, it may or
may not halt (i.e., could loop forever).

• Undecidability:
• Undecidable problems are those that are not recursive
 A Language that is not Recursively Enumerable
Enumerating the binary strings
• We shall need to assign integers to all the binary strings so that each string corresponds to
one integer, and each integer corresponds to one string.
• If w be a binary string, treat 1w as a binary integer i. Then we shall call w the ith string.
• E.g., If w=, then i=1;
• If w=0, then i=2;
• If w=1, then i=3;
• If w=00, then i=4; so on…
• ==> A canonical ordering of all binary strings:
• {, 0, 1, 00, 01, 10, 11, 000, 100, 101, 110, …..}
• {w1, w2, w3, w4, …. wi, … }
• Hereafter, we shall refer to the ith string as wi.
A Language that is not Recursively Enumerable (contd.)
Encoding of a TM:
• A TM can be encoded as a binary string.
• We shall restrict ourselves to TM’s with input alphabet {0, 1}.
• Assign positive integers to the three classes of elements involved in moves:
• Q = {q1, q2, … , qn}

• Γ = {0, 1, ƀ} = {X1, X2, X3}

• Directions D1 (L) and D2 (R).

A Language that is not Recursively Enumerable (contd.)
Encoding of a TM (contd.)
• A move of the TM is represented as:
δ(qi, Xj) = (qk, Xl, Dm).
• Note that we are considering deterministic TMs.
• The above move can be represented as a binary string: 0i10j10k10l10m.
• Thus, each move of the TM can be represented as a binary string.
• A code for the entire TM M consists of all the codes for the transition, in some order,
separated by pairs of 1’s.
• Let this encoding of M be denoted by <M>
• <M> = C111C211C311 …Cn-111Cn
A Language that is not Recursively Enumerable (contd.)
Encoding of a TM (contd.)
• Less Formally:
• Every state, tape symbol, and movement symbol is encoded as a sequence of 0’s:
q1, 0
q2, 00
q3 000
:

0 0
1 00
ƀ 000

L 0
R 00
• Note that 1’s are not used to represent the above, since 1 is used as a special separator symbol.
• Example: δ(q2, 1) = (q3 , 0, R)
Is encoded as: 00100100010100
A Language that is not Recursively Enumerable (contd.)
Encoding of a TM (contd.)
Example: Consider a TM M = ({q1, q2, q3, q4, q5, q6}, {0, 1}, {0, 1, ƀ}, δ , q1, ƀ, {q2}) where δ
consists of the rules.
δ(q1, ƀ)= (q6, ƀ, R)
δ(q6, 1)= (q6, 1, R)
δ(q6, ƀ)= (q3, ƀ, L)
δ(q3, 1)= (q4, ƀ, L)
δ(q4, 1)= (q3, ƀ, L)
δ(q4, ƀ)= (q5, ƀ, R)
δ(q5, ƀ)= (q2, 1, L)
A code for M is:
010001000000100010011000000100100000010010011000000100010001000101100010010
0001000101100001001000100010110000100010000010001001100000100010010010.
A Language that is not Recursively Enumerable (contd.)
Encoding of a TM (contd.)
• If M is a TM, and w is a string, then we shall consider a code pair, (M, w).
• For this pair, we use the code for M followed by 111, followed by w.
• Note that, since no valid code for a TM M contains three 1’s in a row, we can be sure that the first
occurrence of 111 separate the code for M from w.
• For instance, if M were the TM of above example, and w were 1011, then the code for (M, w)
would be the string shown at the end of above example followed by 1111011.
• That is,

0100010000001000100110000001001000000100100110000001000100010001011000100100001000
1011000010010001000101100001000100000100010011000001000100100101111011.
A Language that is not Recursively Enumerable (contd.)
The Diagonalization Language
• We have seen that the set of all TMs over the tape alphabet {0,1, ƀ} can be enumerated
and the set of strings over {0,1} can be enumerated.
• So, we can talk about Mi, the “ith Turing machine”: that TM M whose code is wi, the ith
binary string. The ith integer in binary represents the ith TM.
• Many integers do not correspond to any TM at all.
• For instance, 11001, does not begin with 0, and 001011101001010100 has three
consecutive 1’s.
• If wi is not a valid TM code, we shall take Mi to be the TM with one state and no
transitions. Thus, L(Mi) is Φ if wi fails to be a valid TM code.
A Language that is not Recursively Enumerable (contd.)
The Diagonalization Language (contd.)
• Consider an infinite Boolean matrix.
• The (i, j)th entry is 1 if the ith TM accepts jth string, and 0 otherwise.
• Consider the diagonal elements of this infinite Boolean matrix. Take those w i which
correspond to 0 elements in the matrix.
A Language that is not Recursively Enumerable (contd.)
The Diagonalization Language (contd.)
 Now, we can make a valid definition:
• The language Ld, the diagonalization language, is the set of strings wi such that wi is not in
L(Mi).
Ld= {wi | wi is not accepted by Mi}
• That is, Ld consists of all strings w such that the TM M whose code is w does not accept
when given w as input.
• = {wi | wi is accepted by Mi}
• Now we shall prove that there is NO Turing machine that accepts the language Ld.
A Language that is not Recursively Enumerable (contd.)
Proof that Ld is not Recursively Enumerable
Theorem: Ld is not a recursively enumerable language. There is no Turing machine that accepts the
language Ld
Proof: Suppose Ld is recursively enumerable. Then, it is accepted by a TM M = Mi
• Now, ask if wi is in Ld.
1. If wi  L(Mi) ==> [i, i]=1 ==> wi Ld
2. If wi  L(Mi) ==> [i, i ]=0 ==> wi  Ld
• If wi is in Ld, then Mi accepts wi. But then, by definition of Ld, wi is not in Ld, because Ld contains
only those wj such that Mj does not accept wj.
• Similarly, if wi is not in Ld, then Mi does not accept wi. Thus, by definition of Ld, wi is in Ld.
• Since wi can neither be in Ld nor fail to be in Ld, we conclude that there is a contradiction of our
assumption that M exists. That is, Ld is not a recursively enumerable language.
An Undecidable Problem that is Recursively Enumerable (contd.)
The below Figure suggests the relationship among there classes of languages.
1. Recursive languages
2. The language that are RE but not recursive.
3. The non recursively enumerable (non-RE) language.
An Undecidable Problem that is Recursively Enumerable (contd.)
The Universal Language
• An example of a recursively enumerable, but not recursive language is the language Lu of
a universal Turing machine.
Lu (Universal Language) = {(M, w) | M accepts w}

• That is, Lu is the set of strings representing a TM and an input accepted by that TM. We
shall show that there is a TM U, often called the Universal Turing Machine (UTM), such
that Lu = L (U).
• That is, the UTM takes as input the code for some TM M and some binary string w and
accepts if and only if M accepts w.
• = compliment of Lu.
An Undecidable Problem that is Recursively Enumerable (contd.)
Universal TM
• Universal TM is a TM which can simulate any other TM including itself.
• U has three tapes. The first tape is represented with <M, w>; i.e. the transitions of M are
stored initially on the first tape, along with the string w.
• The second tape initially has a ‘0’, represents initial state q 1. The second tape contains the
information about the state. At any instance M is supposed to be in state q i. While
simulating M, U will have 0i in tape 2.
• Tape 3 is used for simulation.
An Undecidable Problem that is Recursively Enumerable (contd.)
Universal TM (contd.)
The operation of U can be summarized as follows:
1. The UTM checks that M is a valid code for a TM. If M is not valid, its language is empty,
so the UTM immediately halts without accepting.
2. To simulate a move of M, U searches on its first tape for a transition, 0 i10j10k10l10m, such
that 0i is the state on tape 2, and 0j is the tape symbol of M that begins at the position on
tape 3 scanned by U. This transition is the one M would next make. U should:
a) Change the contents of tape 2 to 0k; that is simulate the state change of M.
b) Replace0j on tape 3 by 0l; that is change the tape symbol of M.
c) Move the head on tape 3 to the position of the next 1 to the left or right, respectively,
depending on whether m=1 (move left) or m=2 (move right). Thus U simulates the
move of M to the left or to the right.
3. If M enters its accepting state, then U accepts (i.e., 00 on tape 2).
• In this manner, U simulates M on w. U accepts the code pair (M, w) iff M accepts w.
An Undecidable Problem that is Recursively Enumerable (contd.)
Undecidability of the Universal Language
• The universal language, Lu, is RE but not recursive.
• Lu is accepted by universal TM, so Lu is RE.
• Suppose Lu were recursive, then , the compliment of Lu, would also be recursive.
• However, if we have a TM M to accept , then we can construct a TM to accept Ld.
• Since we already know that Ld is not RE, we have a contradiction of our assumption that Lu is
recursive.
An Undecidable Problem that is Recursively Enumerable (contd.)
Undecidability of the Universal Language (contd.)

Suppose L(M)= . As suggested in above Figure, we can modify TM M into a TM M’ that

accepts Ld as follows.
1. Given string w on its input, M’ changes the input to w111w.
2. M’ simulates M on the new input. If w is wi in our enumeration, then M’ determines
whether Mi accepts wi. Since M accepts , it will accept if and only if M i does not accept
wi; i.e., wi is in Ld.
• Thus, M’ accepts w if and only if w is in Ld. Since we know M’ cannot exist (Ld is not RE),
we conclude that L is not recursive.
 Post’s Correspondence Problem (PCP)
• Post’s Correspondence Problem (PCP) involves strings rather than Turing machines.
• Our goal is to prove this problem about strings to be undecidable, and then use its
undecidability to prove other problems undecidable by reducing PCP to those.
• We shall prove PCP undecidable by reducing Lu to PCP.
• To facilitate the proof, we introduce a “modified” PCP, and reduce the modified problem to
the original PCP.
• Then, we reduce Lu to the modified PCP.
• The chain of reductions is suggested in below Figure. Since the original L u is known to be
undecidable, we conclude that PCP is undecidable.
Post’s Correspondence Problem (contd.)
Definition:
An instance of Post’s Correspondence Problem (PCP) consists of two lists of strings over
some alphabet Ʃ; the two lists must be of equal length. We generally refer to the A and B lists,
and write A = w1, w2, … , wk and B = x1, x2, …, xk, for some integer k. For each i, the pair
(wi,xi) is said to be a corresponding pair.
We say this instance of PCP has a solution, if there is a sequence of one or more
integers i1, i2, …, im that, when interpreted as indexes for strings in the A and B lists, yield the
same string. That is,

We say the sequence i1, i2, …, im is a solution to this instance of PCP, if so.
Post’s Correspondence Problem (contd.)
Example #1
Given an instance of PCP, tell whether this instance has a solution.

Solution:
Post’s Correspondence Problem (contd.)
Example #2
Given an instance of PCP, tell whether this instance has a solution.

Solution:
 The Modified Post’s Correspondence Problem (MPCP)
• It is easier to reduce Lu to PCP if we first introduce an intermediate version of PCP,
which we call the Modified Post’s Correspondence Problem, or MPCP.
Definition:
In the modified PCP, there is the additional requirement on a solution that the first pair on
the A and B lists must be the first pair in the solution. More formally, an instance of MPCP
is two lists A = w1, w2, … , wk and B = x1, x2, …, xk, and a solution is a list of zero or more
integers i1, i2, …, im such that

• In MPCP, the empty list could be a solution if w1= x1.

 The Modified Post’s Correspondence Problem (MPCP)
Example
• The lists of below Figure, may be regarded as an instance of MPCP.

• However, as an instance of MPCP it has no solution.

 The Modified Post’s Correspondence Problem (MPCP)
Reducing MPCP to PCP:
• An important step in showing PCP is undecidable is reducing MPCP to PCP.
Theorem: If PCP were decidable, then MPCP would be decidable (Reducing MPCP to PCP).
Proof:
• We are given an instance of MPCP over an alphabet Ʃ with lists A = w1, w2, … , wk and B =
x1, x2, …, xk.
• We assume * and $ are symbols not present in the alphabet Ʃ of this MPCP instance.
• We construct a PCP instance C = y0, y1, y2, … , yk, yk+1 and D = z0, z1, z2, …, zk, zk+1 as
follows:
1. For i = 1,2,…, k, let yi be wi with a * after each symbol of wi, and let zi be xi with a *
before each symbol of xi.
2. y0 =*y1, and z0=z1.
3. yk+1 = $ and zk+1=*$.
 The Modified Post’s Correspondence Problem (MPCP)
Reducing MPCP to PCP:
Example: An instance of MPCP.
 The Modified Post’s Correspondence Problem (MPCP)
Reducing MPCP to PCP:
• If the constructed instance of PCP has a solution, then the original MPCP instance has a
solution as well.
• We observe that a solution to the PCP instance must begin with index 0 and end with
index k+1, because only the first pair has the same first symbol and the (k+1) th pair has the
same last symbol.
• Thus, the PCP solution can be written 0, i1, i2, …, im, k+1.
• We claim that i1, i2, …, im is a solution to the MPCP instance. The reason is that if we
remove the *’s and the final $ from the string we get the string

• Also, if we remove the *’s and $ from the string we get

• We know that

so it follows that

• Thus, a solution to the PCP instance implies a solution to the MPCP instance.
Completion of the Proof of PCP Undecidability
Reducing Lu to MPCP:
Theorem: If MPCP were decidable, then the problem whether a Turing machine accepts a
string becomes decidable.
Proof: Given a TM M and an input w, we construct an instance of MPCP as follows:
• The essential idea is that MPCP instance (A,B) simulates, in its partial solutions, the
computation of M on input w.
• We may assume our TM never prints a blank, and never moves left from its initial head
position.
• Let M = (Q, Ʃ, Γ, δ, q0, ƀ, F) be a TM, and w be an input string. The MPCP we want to
construct will be over the alphabets Q Γ {#}.
1. The first pair is:
List A List B
# #q0w#
Completion of the Proof of PCP Undecidability (contd.)
Reducing Lu to MPCP (contd.)
2. Tape symbols and the separator # can be appended to both lists.
List A List B
Z Z for each Z in Γ
# #
3. To simulate a move of M, we have certain pairs that reflect those moves. For all q in Q-F
(i.e., q is a nonaccepting state), p in Q, and X, Y, and Z in Γ we have
List A List B
qX Yp if δ(q, X) = (p, Y, R)
ZqX pZY if δ(q, X) = (p, Y, L); Z is any tape symbol
q# Yp# if δ(q, ƀ) = (p, Y, R)
Zq# pZY# if δ(q, ƀ) = (p, Y, L); Z is any tape symbol
Completion of the Proof of PCP Undecidability (contd.)
Reducing Lu to MPCP (contd.)
4. If q is an accepting state, then for all tape symbols X and Y, there are pairs:
List A List B
Z1qZ2 q
Zq q
qZ q
5. We use the following final pair to complete the solution:
List A List B
q## #

• We show that the instance of MPCP, we have constructed, has a solution iff M accepts w.
Completion of the Proof of PCP Undecidability (contd.)
Reducing Lu to MPCP (contd.)
• Example: Consider a TM to accept L = {0n1n | n >= 1}
Next Tape Symbol
Curr. 0 1 X Y ƀ
State
q0 (q1, X, R) - - (q3, Y, R) -
q1 (q1, 0, R) (q2, Y, L) - (q1, Y, R) -
q2 (q2, 0, L) - (q0, X, R) (q2, Y, L) -
q3 - - - (q3, Y, R) (q4, ✔, R)
*q4 - -- - - -
Completion of the Proof of PCP Undecidability (contd.)
Reducing Lu to MPCP (contd.)
Completion of the Proof of PCP Undecidability (contd.)
Reducing Lu to MPCP (contd.)
Completion of the Proof of PCP Undecidability (contd.)
Reducing Lu to MPCP (contd.)
Completion of the Proof of PCP Undecidability (contd.)
Reducing Lu to MPCP (contd.)