‫ر‬

‫الجامعة السعودية االلكتونية‬

‫ر‬
‫االلكتونية‬ ‫الجامعة السعودية‬

‫‪26/12/2021‬‬
Theory of Computing

Headline separator Module 9

Introduction to Turing Machines
Contents
1. Problems That Computers Cannot Solve
2. The Turing Machine
3. Programming Techniques for Turing Machines
4. Extensions to the Basic Turing Machine
Weekly Learning Outcomes
1. Explain Turing machines and their usage as a basic model of
computing.
2. Explain extension to basic Turing machine.
Required Reading
1. Problems That Computers Cannot Solve
2. The Turing Machine
3. Programming Techniques for Turing Machines
4. Extensions to the Basic Turing Machine
(Introduction to Automata Theory, Languages, and Computation
(2013) Global Edition 3rd Edition)
Recommended Reading

https://people.eecs.berkeley.edu/~sseshia/172/lectures/Lecture12.pdf

This Presentation is mainly dependent on the textbook: Introduction to Automata Theory, Languages, and Computation: Global Edition, 3rd edition (2013) PHI
by John E. Hopcroft, Rajeev Motwani and Jeffrey D. Ullman
• Problems That Computers Cannot Solve
Integers, Strings, and Other Things

• Data types have become very important as a

programming tool.
• But at machine level, there is only one type,
integers or strings.
• Key point: Strings that are programs are just
another way to think about the same one data
type.

7
Example: Text

• Strings of ASCII or Unicode characters can be

thought of as binary strings, with 8 or 16
bits/character.
• Binary strings can be thought of as integers.
• It makes sense to talk about “the i-th string.”

8
Binary Strings to Integers

• There’s a small glitch:

• If you think simply of binary integers, then strings like
101, 0101, 00101,… all appear to be “the fifth string.”
• Fix by prepending a “1” to the string before converting
to an integer.
• Thus, 101, 0101, and 00101 are the 13th, 21st, and
37th strings, respectively.

9
Example: Proofs

• A formal proof is a sequence of logical expressions,

each of which follows from the ones before it.
• Encode mathematical expressions of any kind in
Unicode.
• Convert expression to a binary string and then an
integer.
• But a proof is a sequence of expressions, so we
need a way to separate them.
• Also, we need to indicate which expressions are
given.

10
Proofs – (3)

• Quick-and-dirty way to introduce new

symbols into binary strings:
1. Given a binary string, precede each bit by 0.
Example: 101 becomes 010001.
2. Use strings of two or more 1’s as the special
symbols.
Example: 111 = “the following expression is given”;
11 = “end of expression.”

11
 Example: Encoding Proofs

1110100011111100000101110101…

A given End of Expression

expression expression
follows End

A given
An ex- expression
pression follows
Notice this
1 could not
be part of
the “end”

12
Example: Programs

• Programs are just another kind of data.

• Represent a program in ASCII.
• Convert to a binary string, then to an integer.
• Thus, it makes sense to talk about “the i-th
program.”

13
Finite and Infinite Sets
• Intuitively, a finite set is a set for which there is a
particular integer that is the count of the number of
members.
• Example: {a, b, c} is a finite set; its cardinality is 3.
• It is impossible to find a 1-1 mapping between a
finite set and a proper subset of itself.
• Formally, an infinite set is a set for which there is a 1-
1 correspondence between itself and a proper subset
of itself.
• Example: the positive integers {1, 2, 3,…} is an infinite set.
• There is a 1-1 correspondence 1<->2, 2<->4, 3<->6,…
between this set and a proper subset (the set of even
integers).

14
Countable Sets
• A countable set is a set with a 1-1 correspondence with
the positive integers.
• Hence, all countable sets are infinite.
• Example: All integers.
• 0<->1; -i <-> 2i; +i <-> 2i+1.
• Thus, order is 0, -1, 1, -2, 2, -3, 3,…
• Examples: set of binary strings, set of Java programs.
• Examples: Pairs of Integers
• Order the pairs of positive integers first by sum, then by first
component:
• [1,1], [2,1], [1,2], [3,1], [2,2], [1,3], [4,1], [3,2],…, [1,4], [5,1],…
• Interesting exercise: figure out the function f(i, j) such that the
pair [i, j] corresponds to the integer f(i, j) in this order.

15
Enumerations

• An enumeration of a set is a 1-1 correspondence

between the set and the positive integers.
• Thus, we have seen enumerations for strings,
programs, proofs, and pairs of integers.

16
How Many Languages?

• Are the languages over {0,1}* countable?

• No; here’s a proof.
• Suppose we could enumerate all languages over
{0,1}* and talk about “the i-th language.”
• Consider the language L = { w | w is the i-th binary
string and w is not in the i-th language}.

17
Proof – Continued

Lj

Clearly, L is a language over {0,1}*.

x
Thus, it is the j-th language for some
particular j. Recall: L = { w | w is the
i-th binary string and w is
Let x be the j-th string. not in the i-th language}.

Is x in L?
If so, x is not in L by definition of L. j-th

If not, then x is in L by definition of L.

18
Diagonalization Picture

Strings

1 2 3 4 5 …

1 1 0 1 1 0 …

2 1

Languages 3 0

4 0

5 1

… …

19
Diagonalization Picture

Strings Can’t be
Flip each a row –
diagonal 1 2 3 4 5 … it disagrees
entry in an entry
1 0 0 1 1 0 … of each row.

2 0

Languages 3 1

4 1

5 0

… …

20
Proof – Concluded
• We have a contradiction: x is neither in L nor not in L, so
our sole assumption (that there was an enumeration of
the languages) is wrong.
• Comment: This is really bad; there are more languages
than programs.
• E.g., there are languages with no membership algorithm.

21
• The Turing Machine
Turing Machines
• These are very powerful (but abstract) machines
that could simulate any modern day computer.
• Why design such a machine?
• A problem is solved means, for every input,
it answers YES or NO.
• If a problem cannot be solved even using a TM, then it
implies that the problem is undecidable.
• Computability vs. Decidability

23
 A Turing Machine (TM)

• M = (Q, ∑, , , q0, B, F)

Like CPU &

program
Finite counter
Control
Tape is
memory

Tape head
Infinite tape with tape symbols

… B B B X1 X2 X3 … Xi … Xn B B …

Input/output tape symbols

B: blank symbol indicates data boundary)
24
Transition function

• One move (denoted by |---)

in a TM does the following:
X / Y, D
• (q, X) = (p, Y, D) q p
• q is the current state
• X is the current tape symbol below the tape
head
• State changes from q to p
• After the move:
• X is replaced with symbol Y
• D is direction of the head movement
• If D= L, the tape head moves left by one
position.
If D= R, the tape head moves right by one
position.

25
ID of a TM

• Instantaneous Description or ID :
• X1X2…Xi-1qXiXi+1…Xn
means:
• q is the current state
• Tape head is pointing to Xi
• X1X2…Xi-1XiXi+1…Xn are the current tape symbols

• (q,Xi) = (p,Y,R) is same as:

X1…Xi-1qXi…Xn |---- X1…Xi-1YpXi+1…Xn
• (q,Xi) = (p,Y,L) is same as:
X1…Xi-1qXi…Xn |---- X1…pXi-1YXi+1…Xn

26
Way to check for Membership

• Is a string w accepted by a TM?

• Initial condition:
• The (whole) input string w is present in TM,
preceded and followed by infinite blank symbols
• Final acceptance/rejection:
• Accept w if TM enters final state and halts
• If TM halts and not final state, then reject

27
 Example: L = {0n1n | n ≥ 1}
• Strategy: w = 000111

…
B B 0 0 0 1 1 1 B B … B B X X 0 Y Y 1 B B …
… …
…
… B B X 0 0 1 1 1 B B … B B X X X Y Y 1 B B …
… …
… B B X 0 0 Y 1 1 B B … B B X X X Y Y Y B B …
…
… B B X X 0 Y 1 1 B B …
B B X X X Y Y Y B B …

Accept

28
 TM for L = {0n1n | n ≥ 1}

Y / Y, R 1. Mark next unread 0 with X and

0 / 0, R move right
2. Move to the right all the way to
0 / X, R the first unread 1, and mark it
q0 q1 with Y
3. Move back (to the left) all the
Y / Y, R 1 / Y, L way to the last marked X, and
then move one position to the
X / X, R right
q2
Y / Y, R q3 4. If the next position is 0, then
goto step 1.
B / B, R Y / Y, L Else move all the way to the right
0 / 0, L to ensure there are no excess 1s.
q4 If not move right to the next
blank symbol and stop & accept.

29
TM for L = {0n1n | n ≥ 1} using State Diagram
Tabular representation of the state diagram

Next Tape Symbol

Curr.
0 1 X Y B
State
q0 (q1,X,R) - - (q3,Y,R) -

q1 (q1,0,R) (q2,Y,L) - (q1,Y,R) -

q2 (q2,0,L) - (q0,X,R) (q2,Y,L) -

q3 - - - (q3,Y,R) (q4,B,R)

*q4 - -- - - -

30
TMs for calculations

• TMs can also be used for calculating values

• Like arithmetic computations
• E.g., addition, subtraction, multiplication, etc.

31
Example 2: Monus subtraction

“m -- n” = max{m-n,0}
0m10n ➔ ...B 0m-n B.. (if m>n)
...BB…B.. (otherwise)
1. For every 0 on the left (mark X), mark off a 0 on the right
(mark Y)
2. Repeat process, until one of the following happens:
1. // No more 0s remaining on the left of 1
Answer is 0, so flip all excess 0s on the right of 1 to Bs
(and the 1 itself) and halt
2. //No more 0s remaining on the right of 1
Answer is m-n, so simply halt after making 1 to B

32
Example 3: Multiplication

• 0m10n1 (input), 0mn1 (output)

• Pseudocode:
1. Move tape head back & forth such that for every 0
seen in 0m, write n 0s to the right of the last
delimiting 1
2. Once written, that zero is changed to B to get
marked as finished
3. After completing on all m 0s, make the remaining
n 0s and 1s also as Bs

33
• Programming Techniques for Turing Machines
Turing-Machine Theory

• The purpose of the theory of Turing machines is

to prove that certain specific languages have no
algorithm.
• Start with a language about Turing machines
themselves.
• Reductions are used to prove more common
questions undecidable.

35
Picture of a Turing Machine
Action: based on
the state and the
tape symbol under
the head: change
state, rewrite the
symbol and move the
State head one square.

... A B C A D ...

Infinite tape with

squares containing
tape symbols chosen
from a finite alphabet

36
Why Turing Machines?

• Why not deal with C programs or something like

that?
• Answer: You can, but it is easier to prove things
about TM’s, because they are so simple.
• And yet they are as powerful as any computer.
• More so, in fact, since they have infinite memory.

37
Then Why Not Finite-State Machines to Model
Computers?

• In principle, you could, but it is not instructive.

• Programming models don’t build in a limit on
memory.
• In practice, you can go to Fry’s and buy another
disk.
• But finite automata vital at the chip level (model-
checking).

38
Turing-Machine Formalism
• A TM is described by:
1. A finite set of states Q.
2. An input alphabet Σ.
3. A tape alphabet Γ.
4. A transition function δ.
5. A start state q0, in Q.
6. A blank symbol B, in Γ- Σ.
All tape except for the input is blank initially.
7. A set of final states F ⊆ Q.

39
Conventions

• a, b, … are input symbols.

• …, X, Y, Z are tape symbols.
• …, w, x, y, z are strings of input symbols.
• , ,… are strings of tape symbols.

40
The Transition Function

• Takes two arguments:

1. A state, in Q.
2. A tape symbol in Γ.
• δ(q, Z) is either undefined or a triple of the
form (p, Y, D).
• p is a state.
• Y is the new tape symbol.
• D is a direction, L or R.

41
Actions of the PDA

• If δ(q, Z) = (p, Y, D) then, in state q, scanning

Z under its tape head, the TM:
1. Changes the state to p.
2. Replaces Z by Y on the tape.
3. Moves the head one square in direction D.
D = L: move left; D = R; move right.

42
Example: Turing Machine

• This TM scans its input right, looking for a 1.

• If it finds one, it changes it to a 0, goes to final
state f, and halts.
• If it reaches a blank, it changes it to a 1 and
moves left.

43
Example: Turing Machine – (2)

• States = {q (start), f (final)}.

• Input symbols = {0, 1}.
• Tape symbols = {0, 1, B}.
• δ(q, 0) = (q, 0, R).
• δ(q, 1) = (f, 0, R).
• δ(q, B) = (q, 1, L).

44
Simulation of TM

δ(q, 0) = (q, 0, R)
δ(q, 1) = (f, 0, R)
δ(q, B) = (q, 1, L)

... B B 0 0 B B ...

45
Simulation of TM

δ(q, 0) = (q, 0, R)
δ(q, 1) = (f, 0, R)
δ(q, B) = (q, 1, L)

... B B 0 0 B B ...

46
Simulation of TM

δ(q, 0) = (q, 0, R)
δ(q, 1) = (f, 0, R)
δ(q, B) = (q, 1, L)

... B B 0 0 B B ...

47
Simulation of TM

δ(q, 0) = (q, 0, R)
δ(q, 1) = (f, 0, R)
δ(q, B) = (q, 1, L)

... B B 0 0 1 B ...

48
Simulation of TM

δ(q, 0) = (q, 0, R)
δ(q, 1) = (f, 0, R)
δ(q, B) = (q, 1, L)

... B B 0 0 1 B ...

49
Simulation of TM

δ(q, 0) = (q, 0, R)
δ(q, 1) = (f, 0, R)
δ(q, B) = (q, 1, L)

No move is possible.
The TM halts and
... B B 0 0 0 B ... accepts.

50
Instantaneous Descriptions of a Turing
Machine
• Initially, a TM has a tape consisting of a string of
input symbols surrounded by an infinity of blanks in
both directions.
• The TM is in the start state, and the head is at the
leftmost input symbol.

51
 TM ID’s – (2)

• An ID is a string q, where  is the tape between the

leftmost and rightmost nonblanks (inclusive).
• The state q is immediately to the left of the tape symbol
scanned.
• If q is at the right end, it is scanning B.
• If q is scanning a B at the left end, then consecutive B’s at and to the
right of q are part of .

52
TM ID’s – (3)

• As for PDA’s we may use symbols ⊦ and ⊦* to represent

“becomes in one move” and “becomes in zero or more
moves,” respectively, on ID’s.
• Example: The moves of the previous TM are
q00⊦0q0⊦00q⊦0q01⊦00q1⊦000f

53
Formal Definition of Moves

1. If δ(q, Z) = (p, Y, R), then

qZ⊦Yp
If Z is the blank B, then also q⊦Yp
2. If δ(q, Z) = (p, Y, L), then
For any X, XqZ⊦pXY
In addition, qZ⊦pBY

54
Languages of a TM

• A TM defines a language by final state, as usual.

• L(M) = {w | q0w⊦*I, where I is an ID with a final
state}.
• Or, a TM can accept a language by halting.
• H(M) = {w | q0w⊦*I, and there is no move
possible from ID I}.

55
Equivalence of Accepting and Halting

1. If L = L(M), then there is a TM M’ such that

L = H(M’).
2. If L = H(M), then there is a TM M” such that
L = L(M”).

56
Proof of 1: Acceptance -> Halting

• Modify M to become M’ as follows:

1. For each accepting state of M, remove any
moves, so M’ halts in that state.
2. Avoid having M’ accidentally halt.
Introduce a new state s, which runs to the right
forever; that is δ(s, X) = (s, X, R) for all symbols X.
If q is not accepting, and δ(q, X) is undefined, let
δ(q, X) = (s, X, R).

57
Proof of 2: Halting -> Acceptance

• Modify M to become M” as follows:

1. Introduce a new state f, the only accepting
state of M”.
2. f has no moves.
3. If δ(q, X) is undefined for any state q and
symbol X, define it by δ(q, X) = (f, X, R).

58
Recursively Enumerable Languages

• We now see that the classes of languages defined by

TM’s using final state and halting are the same.
• This class of languages is called the recursively
enumerable languages.
• Why? The term actually predates the Turing machine and
refers to another notion of computation of functions.
• An algorithm is a TM that is guaranteed to halt
whether or not it accepts.
• If L = L(M) for some TM M that is an algorithm, we say
L is a recursive language.
• Why? Again, don’t ask; it is a term with a history.

59
Example: Recursive Languages

• Every CFL is a recursive language.

• Use the CYK algorithm.
• Every regular language is a CFL (think of its DFA as
a PDA that ignores its stack); therefore every
regular language is recursive.
• Almost anything you can think of is recursive.

60
Calculations vs. Languages

▪ A calculation is one that takes an input and outputs a value.

▪ The language for a certain calculation is the set of strings of
the form “<input, output>”, where the output corresponds to
a valid calculated value for the input.
▪ A language is a set of strings that meet certain criteria.
e.g., The language Ladd for the addition operation
▪ Membership question is about verifying a solution.
e.g., is “<15#12, 27>” a member of Ladd ?

61
Language of the Turing Machines

• Recursive Enumerable (RE) language

Regular Context-

sensitive

Enumerable
Context

Recursively
(DFA)
free
(PDA)

62
• Extensions to the Basic Turing Machine
 TMs with storage
The standard TMs are equivalent to TMs with storage.
E.g., TM for 01* + 10*

Current Tape Next

Current Storage symbol state
state symbol
q New
Transition function : Storage
storage symbol
• ([q0, B], a) = ([q1, a], a, R)
Tape head
• ([q1,a],a) = ([q1,a], a, R)
B B 0 1 1 1 1 1 B B …
• ([q1,a],B) = ([q2,B], B, R)

where q is current state,

[q, a]:
a is the symbol in storage

64
Multi-track Turing Machines
• A TM with multiple tracks, but just one unified tape head.

control
One tape head to read
k symbols from the k tracks
at one step.

Track 1 … …

… …
Track 2
…
…

Track k … …

66
 Multi-Track TMs

• TM with multiple “tracks” but just one head

• Second track mainly used as a scratch space for marking

• E.g., TM for {wcw | w {0,1}* } but w/o modifying original input string

BEFORE AFTER
control control

Tape head Tape head

… B B 0 1 0 c 0 1 0 B … Track 1 … B B 0 1 0 c 0 1 0 B … Track 1

… B B B B B B B B B B … Track 2 … B B X X X c Y Y Y B … Track 2

67
Multi-tape Turing Machines

• A TM with multiple tapes, each tape with a separate

head
• Each head can move independently of the others

control

k separate heads

Tape 1 … …

… …
Tape 2
…

Tape k … …

70
Non-deterministic TMs
• A TM can have non-deterministic moves:
• (q, X) = { (q1, Y1, D1), (q2, Y2, D2), … }
• Non-deterministic TMs are equivalent to
Deterministic TMs
• Simulation using a multitape deterministic TM:

Control

Input tape
ID1 ID2 ID3 ID4
Marker tape * * * *
Scratch tape

74
Main Reference
1. Problems That Computers Cannot Solve
2. The Turing Machine
3. Programming Techniques for Turing Machines
4. Extensions to the Basic Turing Machine
(Introduction to Automata Theory, Languages, and Computation
(2013) Global Edition 3rd Edition)
Additional References
https://people.eecs.berkeley.edu/~sseshia/172/lectures/Lecture12.pdf

This Presentation is mainly dependent on the textbook: Introduction to Automata Theory, Languages, and Computation: Global Edition, 3rd edition (2013) PHI
by John E. Hopcroft, Rajeev Motwani and Jeffrey D. Ullman
Thank You