Université Paris Cité – LIPADE

Algorithmic Complexity
Turing Machines and Decidability

Jean-Guy Mailly ([email protected])

2023
 Some General Knowledge
Alan Mathison Turing
1

I UK, 1912 – 1954

I Mathematician, computer scientist,
cryptanalyst
I Most famous works:
I Computation model (« Turing
Machines »)
I Work on Enigma during WWII
I Imitation Game (« Turing Test »)
I Father of theoretical CS and AI
I Turing Award ' Nobel Award for CS

J.-G. Mailly | Intro to Complexity

 Outline
2

Turing Machines
Introduction to Turing Machines
Turing Machines and Decision Problems
Non-Determinism
Church-Turing Thesis

Decidability
Recognizable and Decidable Problems
Halting Problem and Reductions

J.-G. Mailly | Intro to Complexity

 Turing Machines
3

Computing Machine [Turing 1936]

I Abstract model to compute any « calculable » decimal number
i.e. any number which can be computed with a finite amount of
resources
I Proposed years before the apparition of computer, but quite a
faithful abstraction of modern machines
I Roughly speaking,
I tape (sequence of squares) browsed by a reader/writer: memory of
a computer
I transition function: processor of a computer

J.-G. Mailly | Intro to Complexity

 Definition of Turing Machines
4

A Turing machine is a tuple hQ, Γ, B, Σ, q0 , δ, F i with:

I Q = {q0 , q1 , . . . , qm }, a finite set of states
I Γ, the finite set of symbols used by the machine (vocabulary)
I B ∈ Γ, a particular symbol called blank
I Σ ⊆ Γ, the input vocabulary
I q0 ∈ Q, the initial state of the machine
I δ : Q × Γ → Q × Γ × {L, R}, the transition function
I F ⊆ Q, the set of final states

J.-G. Mailly | Intro to Complexity

 Transition Function
5

δ : Q × Γ → Q × Γ × {L, R}

I Input: a pair (current state, symbol on the tape) called

configuration
I Output: a tuple (next state, symbol to write, move)

J.-G. Mailly | Intro to Complexity

 Example of Turing Machines (1/2)
Multiplying Integers by 2
6

Let us consider M = hQ, Γ, B, Σ, q0 , δ, F i with:

I Q = {in_progress, done} I q0 = in_progress
I Γ = {0, 1, B} I δ as described in the table
I Σ = {0, 1} I F = {done}

Current state Current symbol Next State Write Move

in_progress 0 in_progress 0 R
in_progress 1 in_progress 1 R
in_progress B done 0 R
done STOP

J.-G. Mailly | Intro to Complexity

 Example of Turing Machines (1/2)
Multiplying Integers by 2
6

Let us consider M = hQ, Γ, B, Σ, q0 , δ, F i with:

I Q = {in_progress, done} I q0 = in_progress
I Γ = {0, 1, B} I δ as described in the table
I Σ = {0, 1} I F = {done}

Current state Current symbol Next State Write Move

in_progress 0 in_progress 0 R
in_progress 1 in_progress 1 R
in_progress B done 0 R
done STOP
Alternative notation of δ On each transition, X /Y /Z is:
0/0/R I X : the current symbol
B/0/R
I Y : the new symbol
start i_p done
I Z : the move
1/1/R

J.-G. Mailly | Intro to Complexity

 Example of Turing Machines (2/2)
Multiplying Integers by 2
7

I Read 1 in state in_progress: write 1, move right, state

in_progress
i_p

1 0 1 0 1 0 B B ...

J.-G. Mailly | Intro to Complexity

 Example of Turing Machines (2/2)
Multiplying Integers by 2
7

I Read 0 in state in_progress: write 0, move right, state

in_progress
i_p

1 0 1 0 1 0 B B ...

J.-G. Mailly | Intro to Complexity

 Example of Turing Machines (2/2)
Multiplying Integers by 2
7

I Read 1 in state in_progress: write 1, move right, state

in_progress
i_p

1 0 1 0 1 0 B B ...

J.-G. Mailly | Intro to Complexity

 Example of Turing Machines (2/2)
Multiplying Integers by 2
7

I Read 0 in state in_progress: write 0, move right, state

in_progress
i_p

1 0 1 0 1 0 B B ...

J.-G. Mailly | Intro to Complexity

 Example of Turing Machines (2/2)
Multiplying Integers by 2
7

I Read 1 in state in_progress: write 1, move right, state

in_progress
i_p

1 0 1 0 1 0 B B ...

J.-G. Mailly | Intro to Complexity

 Example of Turing Machines (2/2)
Multiplying Integers by 2
7

I Read 0 in state in_progress: write 0, move right, state

in_progress
i_p

1 0 1 0 1 0 B B ...

J.-G. Mailly | Intro to Complexity

 Example of Turing Machines (2/2)
Multiplying Integers by 2
7

I Read B in state in_progress: write 0, move right, state done

i_p

1 0 1 0 1 0 B B ...

J.-G. Mailly | Intro to Complexity

 Example of Turing Machines (2/2)
Multiplying Integers by 2
7

I State done: stop

d

1 0 1 0 1 0 0 B ...

J.-G. Mailly | Intro to Complexity

 Accept vs Reject vs Loop
8

We consider a special class of Turing machines, with three possible

« results ». For an input x, we say that the Turing machine M
I accepts x if M reaches the final state YES after a finite number
of steps (i.e. transitions)
I rejects x if M reaches the final state NO after a finite number of
steps
I loops on x if M never reaches a final state

J.-G. Mailly | Intro to Complexity

 Language of a Turing Machine
9

We call language of a Turing Machine M the set of all inputs which

are accepted by M

L(M) = {x ∈ Σ∗ | M stops on x and M(x) = YES}

Example
If M returns YES on inputs P ∈ N[X ] with exactly one root, then
L(M) = {P ∈ N[X ] | P is a polynomial with exactly one root }

J.-G. Mailly | Intro to Complexity

 Language of a Turing Machine
9

We call language of a Turing Machine M the set of all inputs which

are accepted by M

L(M) = {x ∈ Σ∗ | M stops on x and M(x) = YES}

Example
If M returns YES on inputs P ∈ N[X ] with exactly one root, then
L(M) = {P ∈ N[X ] | P is a polynomial with exactly one root }

Language ' Problem

We associate the language L(M) of a Turing Machine M to a
decision problem P

x ∈ L(M) iff x is a positive instance of P

J.-G. Mailly | Intro to Complexity

 (Non-)Deterministic Turing Machines
10

Definition
Given M a Turing machine and δ its transition function, M is
deterministic (DTM) if δ is a mapping from any configuration (q, x) to
(at most) a single image (q 0 , x 0 , m)
Otherwise, M is non-deterministic (NDTM)

J.-G. Mailly | Intro to Complexity

 (Non-)Deterministic Turing Machines
10

Definition
Given M a Turing machine and δ its transition function, M is
deterministic (DTM) if δ is a mapping from any configuration (q, x) to
(at most) a single image (q 0 , x 0 , m)
Otherwise, M is non-deterministic (NDTM)

Solving a Decision Problem

I A DTM solves a decision problem P if the sequence of
transitions leads to the answer for any instance
I Not so easy for NDTM: when the transition function has several
images, each of them must be checked
The machine solves the problem if
I for each positive instance, at least one sequence of transitions
leads to a final state with the answer YES
I for each negative instance, every possible sequence of transitions
leads to a final state with the answer NO
J.-G. Mailly | Intro to Complexity
 Illustration: DTM vs NDTM
11

Deterministic Non-Deterministic
(q0 , x0 ) (q0 , x0 )
δ δ
δ δ
(q1 , x1 ) ... (qk , xk )
(q1 , x1 ) δ
δ δ δ
...
... ...
...
δ δ
δ δ
δ
(qm , xm ) (qn , xn )
(qn , xn ) (qo , xo ) (qp , xp )

J.-G. Mailly | Intro to Complexity

 DTM vs NDTM
12

I Intuitively, NDTM seem to be “stronger” than DTM

J.-G. Mailly | Intro to Complexity

 DTM vs NDTM
12

I Intuitively, NDTM seem to be “stronger” than DTM

I In fact, they the same “computation power”
I Formally: given a problem P, there exists a DTM solving P iff
there exists a NDTM solving P

J.-G. Mailly | Intro to Complexity

 DTM vs NDTM
12

I Intuitively, NDTM seem to be “stronger” than DTM

I In fact, they the same “computation power”
I Formally: given a problem P, there exists a DTM solving P iff
there exists a NDTM solving P
I But using a NDTM can induce shorter time than using a DTM

J.-G. Mailly | Intro to Complexity

 Other Computing Models
13

Other models have been proposed

I Turing machines with several tapes/several dimensions/several
readers-writers
I λ-calculus [Church 1936]
I Kolmogorov-Uspensky machines
[Kolmogorov and Uspensky 1958]
I Schönhage machines [Schönhage 1980]
I ...

Power of Turing machines: the Church-Turing Thesis

Everything which can be computed with any of these models can be
computed with a Turing machine

J.-G. Mailly | Intro to Complexity

 Other Computing Models
13

Other models have been proposed

I Turing machines with several tapes/several dimensions/several
readers-writers
I λ-calculus [Church 1936]
I Kolmogorov-Uspensky machines
[Kolmogorov and Uspensky 1958]
I Schönhage machines [Schönhage 1980]
I ...

Power of Turing machines: the Church-Turing Thesis

Everything which can be computed with any of these models can be
computed with a Turing machine

Turing completeness
A system (computing model, programming language,. . . ) is called
Turing complete if it can compute every possible computable function
J.-G. Mailly | Intro to Complexity
 Outline
14

Turing Machines
Introduction to Turing Machines
Turing Machines and Decision Problems
Non-Determinism
Church-Turing Thesis

Decidability
Recognizable and Decidable Problems
Halting Problem and Reductions

J.-G. Mailly | Intro to Complexity

 Recognizability
15

Recognizability of a Problem
A problem P is called recognizable if there exists a Turing machine
M such that, for each instance i of P, M(i) answers YES iff i is a
positive instance of P

Recognizability of a Language
A language L is called recognizable if there exists a Turing machine
M such that L = L(M)

J.-G. Mailly | Intro to Complexity

 Recognizability
15

Recognizability of a Problem
A problem P is called recognizable if there exists a Turing machine
M such that, for each instance i of P, M(i) answers YES iff i is a
positive instance of P

Recognizability of a Language
A language L is called recognizable if there exists a Turing machine
M such that L = L(M)

Prime Numbers
I Is n ∈ N a prime number?
This problem/language is recognizable: we can write an algorithm
which answers YES when n is prime, and doesn’t care of non-prime
numbers

J.-G. Mailly | Intro to Complexity

 Decidability
16

Decidability of a Problem
A problem P is called decidable if there exists a Turing machine M
such that, for each instance i of P,
I M(i) answers YES when i is a positive instance of P
I M(i) answers NO when i is a negative instance of P

Decidability of a Problem
A language L is called decidable if there exists a Turing machine M
which accepts each x ∈ L and rejects each x ∈ /L
Equivalently: L is decidable iff both L and L̄ are recognizable

J.-G. Mailly | Intro to Complexity

 Decidability
16

Decidability of a Problem
A problem P is called decidable if there exists a Turing machine M
such that, for each instance i of P,
I M(i) answers YES when i is a positive instance of P
I M(i) answers NO when i is a negative instance of P

Decidability of a Problem
A language L is called decidable if there exists a Turing machine M
which accepts each x ∈ L and rejects each x ∈ /L
Equivalently: L is decidable iff both L and L̄ are recognizable

Prime Numbers
I Is n ∈ N a prime number?
This problem/language is decidable: we can write an algorithm which
answers YES when n is prime, and NO otherwise
J.-G. Mailly | Intro to Complexity
 Complement of a Decision Problem
17

Definition
A problem P1 is called the complement of P2 L(P1 ) = L(P2 ).
Equivalently:
I P1 and P2 are defined on the same set of instances
I Positive instances of P1 are negative instances of P2
I Negative instances of P1 are positive instances of P2
Notation: P1 = P2

J.-G. Mailly | Intro to Complexity

 Complement of a Decision Problem
17

Definition
A problem P1 is called the complement of P2 L(P1 ) = L(P2 ).
Equivalently:
I P1 and P2 are defined on the same set of instances
I Positive instances of P1 are negative instances of P2
I Negative instances of P1 are positive instances of P2
Notation: P1 = P2

Prime Numbers
I Given n ∈ N, is there m ∈ N, m 6= 1, m 6= n, s.t. m divides n?
This problem is the complement of the prime number problem.

J.-G. Mailly | Intro to Complexity

 Halting Problem
18

Is there a Turing machine H which has two parameters:

I A Turing machine M
I An input i of M
and which returns
I YES when M stops on i
I NO when M loops on i
If there is such a machine, then Halting problem is decidable;
otherwise, it is undecidable

J.-G. Mailly | Intro to Complexity

 Halting Problem
18

Is there a Turing machine H which has two parameters:

I A Turing machine M
I An input i of M
and which returns
I YES when M stops on i
I NO when M loops on i
If there is such a machine, then Halting problem is decidable;
otherwise, it is undecidable

Result from [Turing 1936]

Halting is undecidable

J.-G. Mailly | Intro to Complexity

 Reducing a Problem to Another
19

Functional Reduction
A functional reduction f is a total computable function from a problem
P1 to a problem P2 such that, for any instance i of P1 , i is a positive
instance iff f (i) is a positive instance of P2
Notation:
P1 ≤f P2

J.-G. Mailly | Intro to Complexity

 Reducing a Problem to Another
19

Functional Reduction
A functional reduction f is a total computable function from a problem
P1 to a problem P2 such that, for any instance i of P1 , i is a positive
instance iff f (i) is a positive instance of P2
Notation:
P1 ≤f P2

Theorem
If P1 ≤f P2 and P1 is undecidable, then P2 is undecidable

J.-G. Mailly | Intro to Complexity

 (Trivial) Example of Functional Reduction
20

I P1 : Given n ∈ N, is n even?
I P2 : Given n ∈ N, is n odd?
I f : N → N is defined by f (n) = n + 1

J.-G. Mailly | Intro to Complexity

 (Trivial) Example of Functional Reduction
20

I P1 : Given n ∈ N, is n even?
I P2 : Given n ∈ N, is n odd?
I f : N → N is defined by f (n) = n + 1
For a positive instance i of P1 , f (i) is a positive instance of P2 .
For a negative instance i of P1 , f (i) is a negative instance of P2 .

J.-G. Mailly | Intro to Complexity

 (Trivial) Example of Functional Reduction
20

I P1 : Given n ∈ N, is n even?
I P2 : Given n ∈ N, is n odd?
I f : N → N is defined by f (n) = n + 1
For a positive instance i of P1 , f (i) is a positive instance of P2 .
For a negative instance i of P1 , f (i) is a negative instance of P2 .
So P1 ≤f P2

J.-G. Mailly | Intro to Complexity

 (Trivial) Example of Functional Reduction
20

I P1 : Given n ∈ N, is n even?
I P2 : Given n ∈ N, is n odd?
I f : N → N is defined by f (n) = n + 1
For a positive instance i of P1 , f (i) is a positive instance of P2 .
For a negative instance i of P1 , f (i) is a negative instance of P2 .
So P1 ≤f P2
Here we also have P2 ≤f P1 ; this is trivial since these are decidable
problems.

J.-G. Mailly | Intro to Complexity

 A Less Trivial Example
21

Language
L1 = {M | M halts on }
with  the empty word. Is L1 decidable?

J.-G. Mailly | Intro to Complexity

 A Less Trivial Example
21

Language
L1 = {M | M halts on }
with  the empty word. Is L1 decidable?

Proof of Undecidability
We need a mapping f : (M, x) 7→ M0 s.t. (M, x) ∈ Halting iff M0 ∈ L1

J.-G. Mailly | Intro to Complexity

 A Less Trivial Example
21

Language
L1 = {M | M halts on }
with  the empty word. Is L1 decidable?

Proof of Undecidability
We need a mapping f : (M, x) 7→ M0 s.t. (M, x) ∈ Halting iff M0 ∈ L1
I M0x (y ) = M(x.y ) with x.y the concatenation of words
I M0x () = M(x), so obviously:
I If M halts on x, then M0x halts on 
I If M loops on x, then M0x loops on 

J.-G. Mailly | Intro to Complexity

 Other Method to Prove Undecidability
22

« Algorithmic » Proof
If P1 is known to be undecidable, we can prove that P2 is undecidable
with an algorithm which computes P1 using only « simple » steps and
calls to P2

J.-G. Mailly | Intro to Complexity

 Other Method to Prove Undecidability
22

« Algorithmic » Proof
If P1 is known to be undecidable, we can prove that P2 is undecidable
with an algorithm which computes P1 using only « simple » steps and
calls to P2

Algorithm 2 Halting
Remember: L1 = {M | Input: M, x = x0 x1 . . . xn
M halts on }. Suppose if x =  then
that ML1 is a Turing return ML1 (M)
Machine which decides else
L1 . x 0 = x1 . . . xn
M0 = M with q0 replaced by δ(q0 , x0 )
return Halting(M0 , x 0 )
end if

J.-G. Mailly | Intro to Complexity

 Other Method to Prove Undecidability
22

« Algorithmic » Proof
If P1 is known to be undecidable, we can prove that P2 is undecidable
with an algorithm which computes P1 using only « simple » steps and
calls to P2

Algorithm 3 Halting
Remember: L1 = {M | Input: M, x = x0 x1 . . . xn
M halts on }. Suppose if x =  then
that ML1 is a Turing return ML1 (M)
Machine which decides else
L1 . x 0 = x1 . . . xn
M0 = M with q0 replaced by δ(q0 , x0 )
return Halting(M0 , x 0 )
end if

J.-G. Mailly | Intro to Complexity

 Other Method to Prove Undecidability
22

« Algorithmic » Proof
If P1 is known to be undecidable, we can prove that P2 is undecidable
with an algorithm which computes P1 using only « simple » steps and
calls to P2

Algorithm 4 Halting
Remember: L1 = {M | Input: M, x = x0 x1 . . . xn
M halts on }. Suppose if x =  then
that ML1 is a Turing return ML1 (M)
Machine which decides else
L1 . x 0 = x1 . . . xn
M0 = M with q0 replaced by δ(q0 , x0 )
return Halting(M0 , x 0 )
end if

J.-G. Mailly | Intro to Complexity

 How to Prove Decidability?
23

I Prove P1 ≤f P2 with P2 decidable, then P1 is decidable

J.-G. Mailly | Intro to Complexity

 How to Prove Decidability?
23

I Prove P1 ≤f P2 with P2 decidable, then P1 is decidable

I Write an algorithm which solves your problem with only
« simple » steps!
Of course, the more complex is the algorithm, the less easy it is to be
sure that it is correct :(

J.-G. Mailly | Intro to Complexity

 How to Prove Decidability?
23

I Prove P1 ≤f P2 with P2 decidable, then P1 is decidable

I Write an algorithm which solves your problem with only
« simple » steps!
Of course, the more complex is the algorithm, the less easy it is to be
sure that it is correct :(

Example: Is n ∈ N a prime number?

Algorithm 7 Prime
Input: n ∈ N √
for x ∈ {2, . . . , b nc} do
if ∃y ∈ N s.t. n = x × y then
return false
end if
end for
return true
J.-G. Mailly | Intro to Complexity
 (Un)Decidability where we don’t expect it
24

I Deciding whether some popular games can be solved may be

impossible

J.-G. Mailly | Intro to Complexity

 (Un)Decidability where we don’t expect it
24

I Deciding whether some popular games can be solved may be

impossible
I “Determining the outcome of a game of Magic: The Gathering in
which all remaining moves are forced is undecidable.”
[Churchill et al 2021]

J.-G. Mailly | Intro to Complexity

 25

[Turing 1936] A. M. Turing, On computable numbers, with an

application to the Entscheidungsproblem. Proceedings of the
London Mathematical Society, 1936.
[Church 1936] A. Church, An unsolvable problem of elementary
number theory. American Journal of Mathematics, 1936.
[Kolmogorov and Uspensky 1958] A. N. Kolmogorov and V.
Uspensky, On the definition of an algorithm. Uspekhi Mat. Naut,
1958.
[Schönhage 1980] A. Schönhage, Storage modification machines.
SIAM Journal on Computing, 1980.
[Churchill et al 2021] A. Churchill, S. Biderman, A. Herrick, Magic:
The Gathering Is Turing Complete. FUN 2021: 9:1-9:19

J.-G. Mailly | Intro to Complexity