LECTURE NOTES

CPSC 351 — Winter 2024

Theoretical Foundations of Computer
Science II
Philipp Woelfel

Chapter 5: Turing Machines

Personal use of this material is permitted. Permission from the author must be obtained for all other
uses, including any form of distribution.
Contents

4 Finite-State Automata 1
4.1 Deterministic Finite-State Automata . . . . . . . . . . . . . . . . . . . . . . . . 1
4.2 Nondeterministic Finite Automata . . . . . . . . . . . . . . . . . . . . . . . . . 6
4.2.1 λ-Transitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4.2.2 NFA Closure of Union . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
4.2.3 NFA Closure of Concatenation . . . . . . . . . . . . . . . . . . . . . . . . 13
4.2.4 NFAs and Regular Expressions . . . . . . . . . . . . . . . . . . . . . . . 15
4.2.5 Converting NFAs to DFAs: The Power Set Construction . . . . . . . . . 19
4.3 Regular Languages and Their Properties . . . . . . . . . . . . . . . . . . . . . . 23
4.3.1 Closure Properties of Regular Languages . . . . . . . . . . . . . . . . . . 24
4.3.2 Proving Nonregularity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

5 Turing Machines 31
5.1 Mathematical Definition of Turing Machines . . . . . . . . . . . . . . . . . . . . 32
5.2 The Language Recognized by a Turing Machines . . . . . . . . . . . . . . . . . . 34
5.3 Turing Machine Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
5.3.1 Memorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
5.3.2 Marking Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
5.4 Turing Machine Variants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
5.4.1 Stay-Put Option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
5.4.2 Multiple Tapes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
5.5 Encoding Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
5.6 Computing Functions with Turing Machines . . . . . . . . . . . . . . . . . . . . 45
5.7 The Universal Turing Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
5.7.1 Encoding Turing Machines . . . . . . . . . . . . . . . . . . . . . . . . . . 45
5.7.2 Constructing the Universal Turing Machine . . . . . . . . . . . . . . . . 46
5.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

ii
5 Turing Machines

Deterministic and non-deterministic finite-state automata are very limited due to the finite
space assumption. Many problems that we can solve with computer programs cannot be solved
with finite space. As we have seen, it is impossible to determine with a DFA if a given string is
of the form 0n 1n , for any n ∈ N.
A more powerful computational model is the Turing machine, named after its inventor, Alan
Turing. Turing machines are powerful enough to solve every problem that can be solved by
modern programming languages, such as Python or Java, assuming that there is no bound on
the available amount of memory. (Strictly speaking, a computer with finite amount of memory
is just a DFA Therefore, the DFA can be viewed as a hardware model, while the Turing machine
is a model for software.)
A Turing machine has a tape for storing information. The tape is divided into an infinite
number of cells, and each cell can store a symbol from some predefined finite tape alphabet, T .
The machine has a read/write head (or just head, for short), which is always located on top of
some tape cell, and can move across the tape. The control unit is similar to a DFA: It is always
in one of a finite number of states from a set Q of states. The machine repeatedly executes the
following steps:
1. The read/write head reads the symbol x ∈ T from the cell it is located on top of.
2. Based on x and its current state q, the machine determines a new state q ′ ∈ Q, a new
symbol x′ ∈ T , and a direction d ∈ {L, R}.
3. The head then writes symbol x′ into the cell at its location, overwriting the previous cell
content, x. (It is possible that x′ = x.)
4. The head moves one cell in the direction indicated by d (left or right).
5. The machine enters the new state q ′ .
Initially, all cells on the tape are blank, except for the input. Blank means that they store a
special blank symbol, ⊥ ∈ T . The input is a string over an input alphabet Σ, where Σ ⊆ T \{⊥}.
That string is initially stored on a number of consecutive cells, and the read/write head is
initially located at the leftmost input cell. This is called the initial configuration. A Turing
machine also has a special state qA , which is the accepting state. Whenever it reaches state qA ,
it halts immediately.

31
 5 Turing Machines

Partial Functions
A partial function f : A → B is almost the same as a function with domain A and range
B, except that there may be some √ element x ∈ A for which δ(x) is not defined. For
example, f : Z → R, where δ(x) = x is a partial function because δ(x) is not defined
for any negative integer x.

5.1 Mathematical Definition of Turing Machines

A Turing machine is uniquely defined by the following components:

• A finite set Q of states.
• A finite input alphabet Σ, which does not contain the blank symbol ⊥.
• A finite tape alphabet T , such that Σ ∪ {⊥} ⊆ T .
• A transition function δ : Q × Σ → Q × Σ × {R, L}, which can be a partial function (see
the box on partial functions).
• An initial state q0 ∈ Q.
• An accepting state qA ∈ Q.
Thus, a Turing machine is defined by a 6-tuple, (Q, Σ, T, δ, q0 , qA ).
The transition function maps each pair (q, x), where q is a state and x is a symbol, to a new
state q ′ , a new symbol x′ , and a direction, d ∈ {L, R}. Thus, δ describes what happens when
the machine reads symbol x while in state q: The head replaces x with x′ at its current location
and moves one step into direction d, and the machine enters state q ′ .
If a Turing machine ever enters a state q and reads a symbol x such that δ(q, x) is undefined,
then it halts immediately, similar as it halts if it enters the accepting state.
The transition function, δ, can be specified in several ways. One way is to give it as a table, as
for example in Figure 5.1. Another option is to list all tuples (q, x, q ′ , x′ , d) such that δ(q, x) =

δ 0 1 ⊥
q0 (q0 , 0, R) (q1 , 1, R) (q3 , ⊥, L)
q1 (q0 , 0, R) (q2 , 1, L) (q3 , ⊥, L)
q2 — (q0 , 0, R) —
q3 — — —

Figure 5.1: The transition function δ of a Turing machine.

32
 5 Turing Machines

(q ′ , x′ , d). For example, the transition function from Figure 5.1 can be given by:

(q0 , 0, q0 , 0, R), (q0 , 1, q1 , 1, R), (q0 , ⊥, q3 , ⊥, L),

(q1 , 0, q0 , 0, R), (q1 , 1, q2 , 0, L), (q1 , ⊥, q3 , ⊥, L), (q2 , 1, q0 , 0, R).

Suppose the initial configuration of a Turing machine with initial state q0 and the above tran-
sition function is
q0
··· ⊥ ⊥ 0 1 1 0 1 1 ⊥ ...

Thus, the input string is 011011, and the head is initially at the leftmost 0. In its first step, the
head reads 0 from the tape. The transition function value for this situation is δ(q0 , 0) = (q0 , 0, R).
Thus, the head replaces the leftmost 0 with another 0 (i.e., the cell content remains the same)
and the head moves one position to the right. This yields the following configuration:
q0
··· ⊥ ⊥ 0 1 1 0 1 1 ⊥ ...

Now the Turing machine reads a 1 from the tape, while it is in state q0 . Since δ(q0 , 1) = (q1 , 1, R),
it does not change the tape contents, moves the head one step to the right, and enters state q1 :
q1
··· ⊥ ⊥ 0 1 1 0 1 1 ⊥ ...

Now the Turing machine is in state q1 and reads 1 from the tape. We have δ(q1 , 1) = (q2 , 1, L),
so the machine does not change the symbol at it’s head position, the head moves to the left,
and the machine enters state q2 :
q2
··· ⊥ ⊥ 0 1 1 0 1 1 ⊥ ...

The machine is now in state q2 , and reads a 1. Since δ(q2 , 1) = (q0 , 0, R), it writes a 0 and moves
the head to the right, resulting in
q0
··· ⊥ ⊥ 0 0 1 0 1 1 ⊥ ...

Continuing this way, we obtain the following sequence of configurations:

33
 5 Turing Machines

q1
··· ⊥ ⊥ 0 0 1 0 1 1 ⊥ ...

q0
··· ⊥ ⊥ 0 0 1 0 1 1 ⊥ ...

q1
··· ⊥ ⊥ 0 0 1 0 1 1 ⊥ ...
q2
··· ⊥ ⊥ 0 0 1 0 1 1 ⊥ ...
q0
··· ⊥ ⊥ 0 0 1 0 0 1 ⊥ ...
q1
··· ⊥ ⊥ 0 0 1 0 0 1 ⊥ ...
q3
··· ⊥ ⊥ 0 0 1 0 0 1 ⊥ ...

At this point, the Turing machine has entered state q3 and its head is positioned over a cell
with symbol 1. Since δ(q3 , 0) is not defined, the Turing machine halts.

5.2 The Language Recognized by a Turing Machines

A Turing machine (Q, Σ, T, δ, q0 , qA ) accepts a string x ∈ Σ∗ , if, starting from the initial con-
figuration for input x, it eventually reaches the accepting state qA . Unlike in the case of finite
automata, a Turing machine may not halt at all for a given input. In that case, it also does not
accept the input. If a Turing machine halts without accepting an input (i.e., it enters a state q
and reads a symbol s such that δ(q, s) is undefined), then it rejects the input.

Definition 5.1. A Turing machine M with input alphabet Σ recognizes the language

L(M ) = {x ∈ Σ∗ | M accepts x}.

Example 5.2. Construct a Turing machine that recognizes the set of bit strings of length at
least two, which have a 1 as their second bit.

34
 5 Turing Machines

The Turing machine is the 6-tuple (Q, Σ, T, δ, q0 , q2 ), where

Q = {q0 , q1 , q2 }, Σ = {0, 1}, T = Σ ∪ {⊥},

and δ is given by the following table:

δ 0 1 ⊥
q0 (q1 , 0, R) (q1 , 1, R) —
q1 — (q2 , 1, R) —
q2 — — —

The Turing machine is initially in state q0 . The transition δ(q0 , ⊥) is not defined; thus, if the first
symbol is blank (i.e., the input is λ), then the Turing machine rejects immediately. Otherwise,
the head moves one position to the right, and enters state q1 . That state indicates that the
head is now positioned on top of the second input symbol. If in state q1 the Turing machine
reads a 1, it enters the accepting state q2 , and thus accepts the input. For any other case, the
transition function is not defined, and thus the machine rejects. ◀

Example 5.3. Construct a Turing machine for the language L = {0n 1n | n ∈ N}.
The machine will proceed by repeatedly executing the following five phases:
1. The head is positioned at the leftmost non-blank symbol on the tape, if such a symbol
exists. If the symbol at the head’s position is blank, then accept (initially, this is only the
case if the input is λ). If the symbol is 1, then reject. Otherwise, it is 0. Overwrite it
with a blank.
2. Sweep to the right, searching for the first blank symbol, then move one step to the left.
3. The head is positioned at the rightmost non-blank symbol on the tape, if such a symbol
exists. If the symbol at the head’s position is blank or 0, then reject. Otherwise, it is 1.
Overwrite it with a blank.
4. Sweep to the left, searching for the first blank symbol, then move one step to the right.
We will implement this by using five states:
• qLE : The machine should be in this state during Phase 1, i.e., when the head is positioned
on the leftmost symbol of the remaining substring of the input.
• qSR : This is used for Phase 2, i.e., when the machine searches for the rightmost non-blank
symbol.
• qRE : The machine should be in this state during Phase 3, i.e., when the head is positioned
on the rightmost non-blank symbol.

35
 5 Turing Machines

δ 0 1 ⊥
qLE (qSR , ⊥, R) — (qacc , ⊥, R)
qSR (qSR , 0, R) (qSR , 1, R) (qRE , ⊥, L)
qRE — (qSL , ⊥, L) —
qSL (qSL , 0, L) (qSL , 1, L) (qLE , ⊥, R)
qA — — —

Figure 5.2: Transition function of a Turing machine recognizing {0n 1n | n ∈ N0 }.

• qSL : In Phase 4, the machine is in this state, while searching for the leftmost non-blank
symbol.
• qA : This is the accepting state.
The Turing machine is the 6-tuple (Q, Σ, T, δ, qLE , qA ), where Q contains the five states above,
Σ = {0, 1}, T = {0, 1, ⊥}, and δ is the transition function depicted in Figure 5.2. Thus, initially
the machine is in state qLE . It is not hard to see, that this realizes the phases described above.
To prove that the machine is correct, we have to show that x ∈ L if and only if the Turing
machine accepts x.
We will show that if x ∈ L, then the Turing machine accepts x. It is also necessary to show that
if the machine accepts an input x, then x ∈ L. We leave that part of the proof as an exercise.
First assume that x ∈ L. Hence, x is of the form 0n 1n . We can prove by induction on n that
the Turing machine accepts x.
The base case is for n = 0. In that case, x = λ. Hence, in the initial configuration the head is on
top of a blank symbol and the machine is in its initial state qLE . Since δ(qLE , ⊥) = (qA , ⊥, R),
the Turing machine reaches the accepting state and accepts the input.
Now let n > 0, and suppose that the Turing machine accepts the input 0n−1 1n−1 . Consider
the input x = 0n 1n . Initially, the machine is in state qLE and on top of the leftmost 0 of the
input. The configuration is shown in (1) of Figure 5.3. All following numbered references in
parentheses refer to Figure 5.3.
The machine now reads a 0, and thus enters state qSR , overwrites the leftmost 0 with a ⊥, and
moves one step to the right. Thus, the tape content is now 0n−1 1n , and the machine is on top of
the first 0 in state qSR —see (2). Since δ(qSR , i) = (qSR , i, R) for each i ∈ {0, 1}, the machine’s
head will now sweep to the right, not changing any of the tape symbols or its state, until it
finds the first blank. Thus, eventually its head is on top of the first blank following the input,
and the state is qSR . This is depicted in (3).
Since δ(qSR , ⊥) = (qRE , ⊥, L), the head now moves one step to the left, and enters state qRE .
Thus, the head is now on top of the rightmost 1 of the remaining string 0n−1 1n on the tape—see
(4). As δ(qRE , 1) = (qSL , ⊥, L), the machine overwrites the rightmost 1 with a blank, enters

36
 5 Turing Machines

qLE
(1) ··· ⊥ 0 0 ... 0 0 1 1 ... 1 1 ⊥ ...
qSR
(2) ··· ⊥ ⊥ 0 ... 0 0 1 1 ... 1 1 ⊥ ...
..
.
qSR
(3) ··· ⊥ ⊥ 0 ... 0 0 1 1 ... 1 1 ⊥ ...
qRE
(4) ··· ⊥ ⊥ 0 ... 0 0 1 1 ... 1 1 ⊥ ...
qSL
(5) ··· ⊥ ⊥ 0 ... 0 0 1 1 ... 1 ⊥ ⊥ ...
..
.
qSL
(6) ··· ⊥ ⊥ 0 ... 0 0 1 1 ... 1 ⊥ ⊥ ...
qLE
(7) ··· ⊥ ⊥ 0 ... 0 0 1 1 ... 1 ⊥ ⊥ ...

Figure 5.3: Configurations of the Turing machine when accepting 0n 1n .

37
 5 Turing Machines

state qSL , and the head moves one step to the left. Now the remaining non-blank string on the
tape is 0n−1 1n−1 . The configuration is shown in (5).
In state qSL , the head sweeps to the left until it finds the first blank (just left of the first 0 of
0n−1 1n−1 ), resulting in the situation depicted in (6). When that happens, because δ(qSL , ⊥) =
(qLE , ⊥, R), the head simply moves one step to the right, and the state changes to qLE .
Now the Turing machine is in state qLE , and its head is on the leftmost position of the remaining
string 0n−1 1n−1 , as depicted in (7). Thus, the machine is in the same configuration, as the initial
one for the input 0n−1 1n−1 . By the inductive hypothesis, the machine accepts 0n−1 1n−1 . Hence,
it accepts.
This proves that if x ∈ L, then the Turing machine accepts the input x. It remains to show
that if the Turing machine accepts x, then x ∈ L. We leave this as an exercise. ◀

5.3 Turing Machine Design

In the previous section, we have given a formal description, of a Turing machine, by specifying
the corresponding 6-tuple. It is not always feasible to do that. In particular, once we understand
how Turing machines work, we will often only give high level descriptions. This means, we will
just on a general level explain how a Turing machine works, but we will not specify, for example,
all state transitions.
In this section, we will present some techniques that Turing machines can use, and that help
with Turing machine design. We will illustrate these techniques through examples. For the first
example, we will provide a formal Turing machine description, and in the second one, a high
level one.

5.3.1 Memorization

The Turing machine can use its state to memorize something about what it has learned from
processing the input. Consider for example the following simple task: Initially, there is only an
input x =∈ {0, 1}∗ on the tape. The goal is to insert the symbol # before the first 1. Thus, if
the input is x = 010110, then the Turing machine should replace this with 0#10110.
We design a machine, which first finds the first 1. We use the state q# , to memorize that it is
currently searching for the first 1 (this should also be the initial state). Once it has found the
first 1, the machine shifts each symbol by one position to the right, and writes # into the free
position. To do that, whenever it reads a symbol, it memorizes that symbol with a special state:
q0 for 0 and q1 for 1. In each step, it reads a symbol, writes the previously memorized symbol
to the tape, memorizes the new symbol, and moves the head one step to the right. Finally, once
the machine reads ⊥, it replaces that symbol with the most recently memorized symbol, and

38
 5 Turing Machines

q#
··· ⊥ 0 1 0 1 1 0 ⊥ ⊥ ...
q#
··· ⊥ 0 1 0 1 1 0 ⊥ ⊥ ...
q1
··· ⊥ 0 # 0 1 1 0 ⊥ ⊥ ...
q0
··· ⊥ 0 # 1 1 1 0 ⊥ ⊥ ...
q1
··· ⊥ 0 # 1 0 1 0 ⊥ ⊥ ...
q1
··· ⊥ 0 # 1 0 1 0 ⊥ ⊥ ...
q0
··· ⊥ 0 # 1 0 1 1 ⊥ ⊥ ...
qhalt
··· ⊥ 0 # 1 0 1 1 0 ⊥ ...

Figure 5.4: Execution of a Turing machine inserting # before the first 1.

enters a state qhalt that forces it to halt (because no transitions are defined). The corresponding
transition function is as follows:
δ 0 1 # ⊥
q# (q# , 0, R) (q1 , #, R) — —
q0 (q0 , 0, R) (q1 , 0, R) — (qhalt , 0, R)
q1 (q0 , 1, R) (q1 , 1, R) — (qhalt , 1, R)
qhalt — — — —
Consider, for example, the input 010110. The execution of the Turing machine is depicted in
Figure 5.4.

5.3.2 Marking Symbols

Another useful technique is to mark symbols. We can do that, by extending the tape alphabet
to have a “marked” version for each regular symbol.

39
 5 Turing Machines

For example, assume that we want to design a Turing machine, that takes an input x ∈ {0, 1}∗ ,
and replaces it with xx on the tape. We will use the “marked” symbolsn e
0 and e1 toomark some
positions of the input. Thus, the tape alphabet of our machine is T = 0, 1, ⊥, e
0, e
1 .

Our Turing machine will work as follows: Recall that initially, the head is positioned on top of
the leftmost input symbol. So if the input is x = x1 . . . xn , then the head is on top of x1 .
In the first phase, the machine will sweep from left to right across the input, replacing each
input symbol with its marked version. Then the head will move back to the first symbol (by
sweeping to the left until it finds a blank, and then going one step to the right). Thus, after
the sweep the tape contents is now xe1 . . . x
fn , where xei is the marked version of xi . The head is
on top of xe1 .
After that, the Turing machine performs the following steps.
1. Sweep left until the first blank is found.
2. Sweep right until either a blank symbol or a marked symbol is found. In case of a blank,
stop. Otherwise, suppose the symbol we found is ea, where a ∈ {0, 1}.
3. Replace e
a with its unmarked version, a.
4. Sweep to the right until the first blank is found.
5. Replace the blank with a.
This is repeated, until the machine stops in the second step.
With each iteration of those steps, the machine replaces the leftmost marked symbol with its
unmarked version, and appends it to the rightmost symbol of the string on the tape.
We leave it as an exercise to give a formal description of the Turing machine.

5.4 Turing Machine Variants

We will now discuss some variants of Turing machines, which make it easier to recognize lan-
guages. For each variant, we will then show that it can be simulated by a regular Turing
machine. Thus, the variants are not more powerful than the regular Turing machine model.

40
 5 Turing Machines

5.4.1 Stay-Put Option

Note
This section was not taught in Winter 2024.

A regular Turing machine must in each step move the head either left or right. For a Turing
machine with stay-put option, this is not the case: In a step, the head may stay in place. On a
formal level, the only difference is that a Turing machine with stay-put option has a transition
function of the form
δ : Q × T → Q × T × {L, R, S},
where Q is the set of states and T the tape alphabet. Thus, it is possible that δ(q, a) = (q ′ , a′ , S),
which means that if the Turing machine reads a in state q, it will enter state q ′ , write a′ , and
the head will not move.
Suppose we have a Turing machine M = (Q, Σ, T, δ, q0 , A) with stay-put option. We can easily
simulate M with a regular Turing machine M . The idea is that whenever M decides to not
move its head, M ′ memorizes the state M is in, moves the head one step to the right, and then
one step back to the left.
For that, we use the state set Q′ = Q × {0, 1}. Thus, each state of M ′ is of the form (q, 0) or
(q, 1), where q is a state of M . A state (q, 0) indicates that M would be in state q with the
head at the same position as the head of M ′ . But M ′ is in state (q, 1), whenever M is in state
q, but the head of M is one cell further to the left than the head of M ′ (because M ′ just moved
its head to the right, whereas M ’s head stayed put).
Thus, if δ ′ is the transition function of M ′ , then

δ ′ (q, 1), a = (q, 0), a, L for each q ∈ A and a ∈ T .

(5.1)

This ensures that if the head of M ′ is one cell too far to the right (as indicated by the 1 in
state (q, 1)), then the head of M ′ will simply move one step to the left. Moreover, if M has a
transition δ(q, a) = (q ′ , a′ , d) defined, where d ∈ {L, R, S}, then M ′ allows the transition
(
(q ′ , 0), a′ , d


′

if d ∈ {L, R}, and
δ (q, 0), a = ′ ′


(q , 1), a , R if d = S.

This means that if M enters state q, writes a′ , and its head moves left or right, then M ′ simulates
this by entering state (q ′ , 0), also writing a′ , and moving the head in the same direction as M .
But if M ’s head stays put, then M ′ transitions into state (q ′ , 1), writes a′ , and moves the head
to the right. In the following step, by eq. (5.1), M ′ will move the head back to the left, and
enter state (q ′ , 0).

41
 5 Turing Machines

5.4.2 Multiple Tapes

A k-tape Turing machine has k tapes instead of one. Each of the tapes has its own read/write
head. In each step, each head reads the symbol at its cell position from the tape. Then, based
on the Turing machine state, and on all k symbols read, the Turing machine now enters a new
state, all heads write a new symbol, and then each head moves independently in one direction
(left or right). Thus, the transition function is now

δ : Q × T k → Q × (T × {L, R})k .

Consider, for example, a Turing machine with 2 tapes. Suppose the first Turing machine head
is on top of a cell with symbol a, and the second head on top of a cell with symbol b. Let q be
the state of the machine. The situation is depicted below:

q
··· ⊥ c c a b a c a b c ⊥ ...

··· ⊥ a b b c c b a b c ⊥ ...

If δ(q, a, b) = q ′ , (c, L), (b, R) , then this means that the machine enters state q ′ , the first head

overwrites the a with a c and moves left, and the second head moves right, not changing the
cell at its position. The result is:

q′

··· ⊥ c c c b a c a b c ⊥ ...

··· ⊥ a b b c c b a b c ⊥ ...

For any k ∈ N, k > 1, we can simulate a k-tape Turing machine M with a regular Turing
machine R. To that end, we encode the contents of the k tapes of M on R’s single tape: If at
some position, the i-th tape of M stores the symbol ai , i ∈ {1, . . . , k}, then at the same position
R’s tape stores the k-tuple (a1 , . . . , ak ). R also needs to keep track of M ’s k head positions.
It does so by marking the corresponding symbols: Suppose the j-th head of M is in position
i, and the tape cell at that position stores the value aj . Then R represents that situation by
marking symbol aj in the tuple (a1 , . . . , ak ) that it stores in position i.
For example, suppose our 2-tape machine M is in the following configuration:

42
 5 Turing Machines

··· ⊥ c c a b a c a b c ⊥ ...

··· ⊥ a b b c c b a b c ⊥ ...

Then the single tape of R should look like this:

··· ⊥ c, a c, b e
a, b b, c a, c c, eb a, a b, b c, c ⊥ . . .

In addition, R always memorizes M ’s state. Then R can simulate a single step of M as follows:
Assume the head of R is on top of the leftmost marked symbol, i.e., in the same position as
the leftmost head of M (as in the picture above). R memorizes the marked symbol and the
tape of M that contains the symbol (in the example above, it would memorize that the first
head of M is on top of symbol a). Then R’s head sweeps towards the right, until it finds the
second marked symbol. Once R has found the second marked symbol, it knows M ’s state and
the symbols at both head positions of M . Thus, R can compute the transition that M applies,
i.e., the symbols M writes, the head movements, and M ’s new state. Using another sweep, R
can update its tape accordingly.
By repeatedly simulating every single step of M , R can simulate the entire execution of M on
a given input.

5.5 Encoding Objects

We are interested in Turing machines performing computations on objects, such as numbers,

sets, graphs, or other Turing machines. Any object we typically deal with can be encoded by a
bit string, or a string from some larger finite alphabet. We will use angled brackets to indicate
the encoding of an object over some finite alphabet Σ (which is defined by the context): If O is
an object, then ⟨O⟩ denotes the encoding of O.
For example, a non-negative integer x ∈ N0 can be encoded using a bit-string of x 0s

. . 0} = 0x .
⟨x⟩ = |0 .{z
x

This is called the unary encoding. Obviously, this is not a very (space) efficient encoding, and
for most applications binary encoding is more suitable.

43
 5 Turing Machines

A non-negative integer x ∈ N0 has the binary encoding ⟨x⟩2 = bn−1 . . . b0 ∈ {0, 1}n ,

b0 · 20 + b1 · 21 · · · + bn−1 2n−1 = x.

Note that this encoding is not unique, because adding a string of 0s as a prefix does not change
the value of the encoded integer. However, we can easily make it unique by requiring that there
are no leading 0s. (Note that then the integer 0 is then encoded by the empty string λ, i.e.,
⟨0⟩2 = λ.)
We can encode multiple objects by encoding each individual object separately, and then ap-
pending the individual codes separated by a special symbol. For example, a set {x1 , . . . , xk } of
non-negative integers can be encoded by a string over {0, 1, #}, by using the binary encoding
for each integer xi , and using # to separate integers:

⟨x1 , . . . , xk ⟩ = ⟨x1 ⟩ # ⟨x2 ⟩ # . . . # ⟨xk ⟩ .

For example, the set {1, 3, 4} could be encoded as follows:

1#11#100#1001.

Observe that any code using three or more symbols can easily be changed to a binary code, i.e.,
a code using only two symbols. For example, consider a code that uses the symbols {0, 1, #}, as
above. In an encoding of an object, we can replace each 0 with 00, each 1 with 11, and each #
with 01. For example, the encoding of the two sets {1, 3} and {2, 4} from above would become

⟨{1, 3, 4, 9}⟩ = 11 01 11 11 01 11 00 00 01 11 00 00 11.

Often we want to encode more complicated objects. For example, we may want to encode
a directed graph G = (V, E), where V is the set of vertices and E the set of edges. If we
don’t care about the vertex labels (which is usually the case), then we may assume that V =
{0, 1, . . . , k − 1} for some k ∈ N0 . Thus, we can encode V as ⟨k⟩2 . Each edge e = (u, v) ∈ E
is then a pair of integers u, v ∈ {0, . . . , k − 1}, and we can encode it as ⟨u⟩2 # ⟨v⟩2 . To encode
the entire graph, we simply write down the code of V followed by # and then the code for E.
For example, the directed triangle 0 → 1 → 2 → 0 would be encoded as

⟨3⟩2 # ⟨0⟩2 # ⟨1⟩2 # ⟨1⟩2 # ⟨2⟩2 # ⟨2⟩2 # ⟨0⟩2 = 11##1#1#10#10#.

(Recall that the empty string encodes 0, i.e., ⟨0⟩2 = λ.)

44
 5 Turing Machines

5.6 Computing Functions with Turing Machines

Consider a Turing machine M with input alphabet Σ and tape alphabet T . Then M computes
a partially defined function f : Σ∗ → T ∗ . If M does not halt for some input x ∈ Σ∗ , then
f (x) is not defined. But if M halts, then f (x) is the contents of the tape between the leftmost
non-blank symbol and the rightmost non-blank symbol.
Suppose, for example, a Turing machine halts for some input x in the following configuration:

q
··· ⊥ ⊥ 0 1 0 ⊥ 2 0 2 ⊥ ⊥ ...

Then the function f computed by that machine satisfies f (x) = 010⊥202.

Example 5.4. Construct a Turing machine that computes the partially defined function f :
{0, 1}∗ → {0, 1}∗ , where for any two non-negative integers f (⟨x⟩ ⟨y⟩) = f (⟨x + y⟩). Assume the
unary encoding of an integer x, where ⟨x⟩ = 0x 1.
Given an input 0x 10y 1, the Turing machine can simply shift the first x 0s by one position to
the right, as in Section 5.3.1. Then it halts, and thus its output is 0x+y 1. ◀

5.7 The Universal Turing Machine

We will now discuss a Turing machine that can simulate other Turing machines. This Universal
Turing machine takes as input a description of another Turing machine, say M , as well as an
input w, and then it simulates M on w. Thus, the Universal Turing machine is similar to a
Python interpreter written in Python, which takes as input a Python program and then executes
it.

5.7.1 Encoding Turing Machines

There are many ways of encoding Turing machines using a finite alphabet, but for the sake of
concreteness, we will describe one particular encoding. We will use the alphabet Σ = {0, 1}.
Recall that a Turing machine is a 6-tuple (Q, Σ, T, δ, q0 , qA ). We can assume that Q = {0, . . . , q}
for some integer q, Σ = {0, . . . , s} for some integer s, and T = {0, . . . , t} for some integer t > s.
The blank symbol is t, and thus it is not in Σ as required. Hence, to encode Q, Σ, and T , we
just need to encode the integers q, s, and t.

45
 5 Turing Machines

The transition function, δ, is a list of quintuples: The transition δ(q, a) = (q ′ , a′ , d) for q, q ′ ∈ Q,

a, a′ ∈ T and d ∈ {L, R}, corresponds to a quintuple (q, a, q ′ , a′ , d). For d we just use the
numbers 0 instead of L and 1 instead of R. Then each element of the quintuple (q, a, q ′ , a′ , d)
is a number. Thus, the entire transition function can be encoded by a sequence of numbers.
Finally, the starting state, q0 ∈ Q is a number in {0, . . . , q}, and the set of accepting states
is also just a set of numbers. Thus, we can encode each component of the Turing machine by
encoding a number or a sequence of numbers. We can separate the codes for each component
with the symbol #. Finally, replacing each 0 with 00, each 1 with 11, and each # with 01 (as
discussed in Section 5.5), we obtain an encoding of the Turing machine that uses the alphabet
{0, 1}.

5.7.2 Constructing the Universal Turing Machine

Let Σ be some finite alphabet of size at least 2, which does not contain #. The Universal Turing
machine, U , takes as input a string ⟨M ⟩ #w, where ⟨M ⟩ ∈ Σ∗ is the encoding of a (single tape)
Turing machine M with input alphabet Σ, and w ∈ Σ∗ .
We assume that U has three tapes (we know from Section 5.4.2, that we can then construct an
equivalent single tape machine from U ). Now, U simulates M on the input w as follows: Recall
that the input to U is ⟨M ⟩ #w. First, U copies w to the second tape. Then it moves the head
of the second tape to the left end of w. Finally, U searches through the description of M for
the code of M ’s initial state. It copies that to the third tape.
From now on, U ’s second tape will always look like M ’s tape in the simulated execution. Also,
U ’s second head position will always be in the same position as M ’s head in the simulated
execution. On the third head, U will store the state that M is in.
To simulate one step of M , U ’s first head searches for the description of the transition that M
would apply. To do that, U can find M ’s simulates state on the third tape, and the symbol that
M would read on the second tape. Once U has found the description of M ’s next transition,
it can apply it by updating the second and third tape accordingly. I.e., the second head writes
what M would write and moves in the same direction as M ’s head would, and the third head
writes M ’s new state onto the tape. This way, U can simulate M step by step, until either M
accepts or rejects (and then U can accept or reject, too).

46
 5 Turing Machines

5.8 Exercises

5.1 Let M be a Turing machine (Q, Σ, T, δ, q0 , q2 ), where Q = {q0 , q1 , q2 }, Σ = Turing machines

{0, 1}, T = {0, 1, ⊥}, and δ is defined by the following table:
δ 0 1 ⊥
q0 (q1 , 1, R) (q1 , 0, R) (q1 , 0, R)
q1 (q2 , 1, L) (q1 , 0, R) (q2 , 0, L)
q2 — — —

For each of the following inputs, determine the final tape contents (when the
Turing machine halts):
(a) 0011
(b) 101
(c) 11.
5.2 Let M be a Turing machine (Q, Σ, T, δ, q0 , q2 ), where Q = {q0 , q1 , q2 }, Σ = Turing machines
{0, 1}, T = {0, 1, ⊥}, and δ is defined by the following table:
δ 0 1 ⊥
q0 (q0 , 0, R) (q1 , 0, R) (q2 , ⊥, R)
q1 (q1 , 0, R) (q0 , 1, R) (q2 , ⊥, R)
q2 — — —
Describe the behaviour of M in case the input is
(a) 1k for some k ∈ N;
(b) an arbitrary string in {0, 1}∗ .
5.3 Construct a Turing machine (by specifying the corresponding 6-tuple) with Turing machines
tape alphabet T = {0, 1, ⊥}, which for an input x ∈ {0, 1}∗
(a) replaces the first 1 in x with a 0 and does not change any other tape
symbols;
(b) appends a 1 to the end of x and does not change any other tape symbols;
(c) replaces the first occurrence of 01 in x with 10 and does not change any
other tape symbols;

47
 5 Turing Machines

(d) halts in an accepting state if and only if x contains the sub-string 010.
5.4 For each of the following languages construct a Turing machine that recognizes Turing machines
it. Give high level and formal descriptions (in form of 6-tuples).
(a) L1 = {x ∈ {0, 1}∗ | x contains an even number of 0s};
(b) L2 = {x ∈ {a, b}∗ | x contains equally many a’s and b’s};
(c) L3 = {a2n bn | n ≥ 0};
5.5 Give a formal description of a Turing machine that accepts the language
L = {w#x | w, x ∈ {a, b}∗ and w is a substring of x}.

5.6 Consider a variant of the Turing machine model, where the tape is bounded to Turing machine variants
the left (but still has infinite length to the right). If the head is in the leftmost
position, and the transition functions requires the head to move left, then the
head simply stays put. Initially the input is written from left to right starting
at the leftmost cell. Show that such a Turing machine can simulate a regular
Turing machine, which has a doubly infinite tape.
5.7 Now consider a variant of the Turing machine model, where the tape is bounded Turing machine variants
to the left as in the previous question. In addition, the head cannot move a
single position to the left. Instead, it can either only move one position to the
right, or it “jumps” to the leftmost position (we call this a head reset). Hence,
the transition function is of the form
δ : Q × T → Q × T × {R, RESET }.
Show that such a Turing machine can simulate a regular Turing machine and
vice versa.
5.8 Complete the proof of Example 5.3: Show that if the Turing machine from Proofs, Turing machines
that example accepts a string x, then x ∈ {0n 1n | n ∈ N0 }.
5.9 Give a formal description of the Turing machine from Section 5.3.2, which Turing machines
replaces the input x ∈ {0, 1}∗ with xx.
5.10 For a non-negative integer x ∈ N let ⟨x⟩2 denote the binary encoding of x. Functions computed by Turing machines
For Σ = {0, 1, #}, let f : Σ∗ → Σ∗ be the partial function that maps each
string ⟨x⟩2 # ⟨y⟩2 , where x, y ∈ N, to ⟨x + y⟩2 . Give a Turing machine that
computes f . (If f (w) is not defined for some input w ∈ Σ∗ , then the output
can be arbitrary.)

48
 5 Turing Machines

5.11 Describe a multi-tape Turing machine that computes for each integer x ∈ N Functions computed by Turing machines
the function f (⟨x⟩2 ) = ⟨2x ⟩2 . You may assume that the Turing machine has
special states, which it can enter, upon which it computes on one of its tapes
one of the following functions without changing any of the other tapes:
• add, where add(⟨a⟩2 # ⟨b⟩2 ) = ⟨a + b⟩2 for a, b ∈ N.
• sub, where sub(⟨a⟩2 # ⟨b⟩2 ) = ⟨a − b⟩2 for a, b ∈ N.
• mult, where mult(⟨a⟩2 # ⟨b⟩2 ) = ⟨a · b⟩2 for a, b ∈ N.
• comp, where comp(⟨a⟩2 # ⟨b⟩2 ) = 1 if a < b and comp(⟨a⟩2 # ⟨b⟩2 ) = 0,
otherwise.
5.12 A write-once Turing machine is a single-tape Turing machine that can alter Turing machine variants
each tape cell at most once (including the input portion of the tape). Show that
any standard Turing machine has an equivalent write-once Turing machine.
Hint: As a first step you may try to simulate M by a “write-twice” T M . Also,
you will need to use a lot of tape space for the simulation.
5.13 Note that a Turing machine must have exactly one accepting state. But similar Turing machine variants
as for DFAs, we could define Turing machines with multiple accepting states.
Prove that for any Turing machine M with multiple accepting states, there is
an equivalent Turing machine M ′ that has exactly one accepting state. I.e.,
for each input x, M ′ halts if and only if M halts, and M ′ accepts if and only
if M accepts.
5.14 Describe encoding schemes for the following objects, using the alphabet Encoding objects
{0, 1, #}.
(a) A fraction, i.e., a number in Q.
(b) A directed graph, where each edge has an associated weight, which is a
non-negative integer. (Weights can be used to describe distances between
endpoints, if the graph represents geographical locations and connections
between them.)
5.15 Describe an encoding scheme for DFAs using the alphabet {0, 1, #}. Encode Encoding Objects
the following DFA using your scheme:

49
 5 Turing Machines

0 1
0
q0 1 q1 q2

0,1

50