Chapter 8

Complexity

231
 8.1 Introduction

• Solvable problems versus efficiently solvable problems.

• Measuring complexity: complexity functions.

• Polynomial complexity.

• NP-complete problems.

232
 8.2 Measuring complexity

• Abstraction with respect to the machine being used.

• Abstraction with respect to the data (data size as only parameter).

• O notation.

• Efficiency criterion: polynomial.

233
 8.3 Polynomial problems

• Influence of the encoding.

• Graph example.

• Reasonable encodings:

– no padding,
– polynomial decoding,
– unary representation of numbers not allowed.

234
 Complexity and Turing machines

Time complexity of a Turing machine that always stops:

TM (n) = max {m | ∃x ∈ Σ∗, |x| = n and the execution of M on x
is m steps long}.

A Turing machine is polynomial if there exists a polynomial p(n) such that

TM (n) ≤ p(n)
for all n ≥ 0.

The class P is the class of languages that are decided by a polynomial

Turing machine.

235
 8.4 Polynomial transformations

• Diagonalisation is not adequate to prove that problems are not in P.

• Another approach: comparing problems.

236
 The Travelling Salesman (TS)

• Set C of n Cities.

• Distances d(ci, cj ).

• A constant b.

• Is there a permutation of the towns such that:

X
d(cpi , cpi+1 ) + d(cpn , cp1 ) ≤ b.
1≤i<n

237
 Hamiltonian Circuit (HC)

• Graph G = (V, E)

• Is there a closed circuit in the graph that contains each vertex exactly
once.

s s s s
@ @
@ @
@s
@ @
@
@ @
@ @
s @s
@ s @s
@

238
 Definition of polynomial transformations

Goal : to establish a link between problems such as HC and TS (one is in

P if and only if the other is also in P).

Definition :
Consider languages L1 ∈ Σ∗1 and L2 ∈ Σ∗2. A polynomial transformation
from L1 to L2 (notation L1 ∝ L2) is a function f : Σ∗1 → Σ∗2 that satisfies
the following conditions :

1. it is computable in polynomial time,

2. f (x) ∈ L2 if and only if x ∈ L1.

239
 HC ∝ TS

• The set of cities is identical to the set of vertices of the graph, i.e.
C =V.

(
1 si (ci, cj ) ∈ E
• The distances are the following (ci, cj ) = .
2 si (ci, cj ) 6∈ E

• The constant b is equal to the number of cities, i.e. b = |V |.

240
 Properties of ∝

If L1 ∝ L2, then

• if L2 ∈ P then L1 ∈ P,

• if L1 6∈ P then L2 6∈ P.

If L1 ∝ L2 et L2 ∝ L3, then

• L1 ∝ L3.

241
 Polynomially equivalent problems

Definition
Two languages L1 and L2 are polynomially equivalent (notation L1 ≡P L2)
if and only if L1 ∝ L2 and L2 ∝ L1.

• Classes of polynomially equivalent problems: either all problems in the

class are in P, or none is.

• Such an equivalence class can be built incrementally by adding

problems to a known class.

• We need a more abstract definition of the class containing HC and TS.

242
 The class NP

• The goal is to characterise problems for which it is necessary to

examine a very large number of possibilities, but such that checking
each possibility can be done quickly.

• Thus, the solution is fast, if enumerating the possibilities does not

cost anything.

• Modelisation : nondeterminism.

243
 The complexity of nondeterministic Turing machines

The execution time of a nondeterministic Turing machine on a word w is

given by

• the length of the shortest execution accepting the word, if it is

accepted,

• the value 1 if the word is not accepted.

The time complexity of M (non deterministic) is the function TM (n)

defined by
TM (n) = max {m | ∃x ∈ Σ∗, |x| = n and
the execution time of M on x is m steps long}.

244
 The definition of NP

Définition
The class NP (from Nondeterministic Polynomial) is the class of languages
that are accepted by a polynomial nondeterministic Turing machine.

Exemple
HC and TS are in NP.

245
Theorem
Consider L ∈ NP. There exists a deterministic Turing machine M and a
polynomial p(n) such that M decides L and has a time complexity
bounded by 2p(n).

Let Mnd be a nondeterministic machine of polynomial complexity q(n)

that accepts L. The idea is to simulate all executions of Mnd of length
less than q(n). For a word w, the machine M must thus:

1. Determine the length n of w and compute q(n).

2. Simulate each execution of Mnd of length q(n) (let the time needed be
q 0(n)). If r is the largest number of possible choices within an
execution of Mnd, there are at most rq(n) executions of length q(n).

246
3. If one of the simulated executions accepts, M accepts. Otherwise, M
stops and rejects the word w.

0
Complexity : bounded by rq(n) × q 0(n) and thus by 2log2(r)(q(n)+q (n)),
which is of the form 2p(n).

247
 The structure of NP

Definition A polynomial equivalence class C1 is smaller than a polynomial

equivalence class C2 (notation C1  C2) if there exists a polynomial
transformation from every language in C1 to every language in C2.

Smallest class in NP : P

• The class NP contains the class P (P ⊆ NP).

• The class P is a polynomial equivalence class.

• For every L1 ∈ P and for every L2 ∈ NP, we have L1 ∝ L2.

248
Largest class in NP : NPC

A language L is NP-complete if

1. L ∈ NP,

2. for every language L0 ∈ NP, L0 ∝ L.

Theorem
If there exists an NP-complete language L decided by a polynomial
algorithm, then all languages in NP are polynomially decidable, i.e.
P = NP.

Conclusion : An NP-complete problem does not have a polynomial

solution if and only if P 6= NP

249
 NPC

NP

250
 Proving NP-completeness

To prove that a language L is NP-complete, one must establish that

1. it is indeed in the class NP (L ∈ NP),

2. for every language L0 ∈ NP, L0 ∝ L,

or, alternatively,

3. There exists L0 ∈ NPC such that L0 ∝ L.

Concept of NP-hard problem.

251
 A first NP-complete problem
propositional calculus

Boolean calculus :

p q p∧q
p ¬p 0 0 0
0 1 0 1 0
1 0 1 0 0
1 1 1

p q p∨q p q p⊃q
0 0 0 0 0 1
0 1 1 0 1 1
1 0 1 1 0 0
1 1 1 1 1 1

252
• Boolean expression: (1 ∧ (0 ∨ (¬1))) ⊃ 0.

• Propositional variables and propositional calculus :

(p ∧ (q ∨ (¬r))) ⊃ s.

• Interpretation function. Valid formula, satisfiable formula.

• Conjunctive normal form: conjunction of disjunctions of literals.

253
 Cook’s theorem

SAT Problem : satisfiability of conjunctive normal form propositional

calculus formulas.

Theorem
The SAT problem is NP-complete

Proof

1. SAT is in NP.

2. There exists a polynomial transformation from every language in NP

to LSAT.

• Transformation with two arguments : word and language.

• The languages of NP are characterised by a polynomial-time
nondeterministic Turing machine.

254
Word w (|w| = n) and nondeterministic polynomial Turing machine
M = (Q, Γ, Σ, ∆, s, B, F ) (bound p(n)).

Description of an execution of M (T : tape; Q : state; P : position; C :

choice.)

Q P C T


 ···
···





···



... ... ... ...

p(n) + 1
···





···





···


| {z }
p(n)+1

255
Representing an execution with propositional variables:

1. A proposition tijα for 0 ≤ i, j ≤ p(n) and α ∈ Γ.

2. A proposition qiκ for 0 ≤ i ≤ p(n) and κ ∈ Q.

3. A proposition pij for 0 ≤ i, j ≤ p(n).

4. A proposition cik for 0 ≤ i ≤ p(n) and 1 ≤ k ≤ r.

256
Formula satisfied only by an execution of M that accepts the word w :
conjunction of the following formulas.

 
^  _ ^
( tijα) ∧ (¬tijα ∨ ¬tijα0 )


0≤i,j≤p(n) α∈Γ α6=α0 ∈Γ

One proposition for each tape cell. Length O(p(n)2).

 
^ _ ^
( pij ) ∧ (¬pij ∨ ¬pij 0 )
 

0≤i≤p(n) 0≤j≤p(n) 0≤j6=j 0 ≤p(n)

One proposition for each position. Length O(p(n)3).

257
  
^ ^
t0jwj+1 ∧ t0jB  ∧ q0s ∧ p00
 

0≤j≤n−1 n≤j≤p(n)

Initial state. Length O(p(n))

^
[(tijα ∧ ¬pij ) ⊃ t(i+1)jα]
0≤i<p(n)
0≤j≤p(n)
α∈Γ

^
[¬tijα ∨ pij ∨ t(i+1)jα]
0≤i<p(n)
0≤j≤p(n)
α∈Γ

Transitions, tape not modified. Length O(p(n)2).

258
  
((qiκ ∧ pij ∧ tijα ∧ cik ) ⊃ q(i+1)κ0 )∧
^ 
 ((qiκ ∧ pij ∧ tijα ∧ cik ) ⊃ t

 (i+1)jα0 )∧ 

0≤i<p(n) ((qiκ ∧ pij ∧ tijα ∧ cik ) ⊃ p(i+1)(j+d))
0≤j≤p(n)
α∈Γ
1≤k≤r

 
(¬qiκ ∨ ¬pij ∨ ¬tijα ∨ ¬cik ∨ q(i+1)κ0 )∧
^ 
 (¬qiκ ∨ ¬pij ∨ ¬tijα ∨ ¬cik ∨ t

 (i+1)jα0 )∧ 

0≤i<p(n) (¬qiκ ∨ ¬pij ∨ ¬tijα ∨ ¬cik ∨ p(i+1)(j+d))
0≤j≤p(n)
α∈Γ
1≤k≤r

Transitions, modified part. Length O(p(n)2).

259
 _
[qiκ]
0≤i≤p(n)
κ∈F

Final state reached. Length O(p(n)).

• Total length of the formula O(p(n)3).

• The formula can be built in polynomial time.

• Thus, we have a transformation that is polynomial in terms of n = |w|.

• The formula is satisfiable if and only if the Turing machine M accepts.

260
 Other NP-complete problems

3-SAT : satisfiability for conjunctive normal form formulas with exactly 3

literals per clause.

SAT ∝ 3-SAT.

1. A clause (x1 ∨ x2) with two literals is replaced by

(x1 ∨ x2 ∨ y) ∧ (x1 ∨ x2 ∨ ¬y)

2. A clause (x1) with a single literal is replaced by

(x1 ∨ y1 ∨ y2) ∧ (x1 ∨ y1 ∨ ¬y2) ∧
(x1 ∨ ¬y1 ∨ y2) ∧ (x1 ∨ ¬y1 ∨ ¬y2)

261
3. A clause
(x1 ∨ x2 ∨ · · · ∨ xi ∨ · · · ∨ x`−1 ∨ x`)
with ` ≥ 4 literals is replaced by
(x1 ∨ x2 ∨ y1) ∧ (¬y1 ∨ x3 ∨ y2)
∧ (¬y2 ∨ x4 ∨ y3) ∧ · · ·
∧ (¬yi−2 ∨ xi ∨ yi−1) ∧ · · ·
∧ (¬y`−4 ∨ x`−2 ∨ y`−3)
∧ (¬y`−3 ∨ x`−1 ∨ x`)

262
The vertex cover problem (VC) is NP-complete.

Given a graph G = (V, E) and an integer j ≤ |V |, the problem is to

determine if there exists a subset V 0 ⊆ V such that |V 0| ≤ j and such that,
for each edge (u, v) ∈ E, either u, or v ∈ V 0.
s s s
@
@
@
@
@
@
s @s
@ s

sf sf sf
@
@
@
@
@
@
sf @s
@ s

263
3-SAT ∝ VC

Instance of 3-SAT :
E1 ∧ · · · ∧ Ei ∧ · · · ∧ Ek
Each Ei is of the form
xi1 ∨ xi2 ∨ xi3
where xij is a literal. The set of propositional variables is

P = {p1, . . . , p`}.

The instance of VC that is built is then the following.

264
1. The set of vertices V contains

(a) a pair of vertices labeled pi and ¬pi for each propositional variable
in P,

(b) a 3-tuple of vertices labeled xi1, xi2, xi3 for each clause Ei.

The number of vertices is thus equal to 2` + 3k.

265
2. The set of edges E contains

(a) The edge (pi, ¬pi) for each pair of vertices pi, ¬pi, 1 ≤ i ≤ `,

(b) The edges (xi1, xi2), (xi2, xi3) et (xi3, xi1) for each 3-tuple of
vertices xi1, xi2, xi3, 1 ≤ i ≤ k,

(c) an edge between each vertex xij and the vertex p or ¬p representing
the corresponding literal.

The number of edges is thus ` + 6k.

3. The constant j is ` + 2k.

266
Example

(p2 ∨ ¬p1 ∨ p4) ∧ (¬p3 ∨ ¬p2 ∨ ¬p4)

ps 1 ¬p
s 1
ps 2 ¬p
s 2
ps 3 ¬p
s 3
ps 4 ¬p
s 4
E Z
E  Z 
 Z 
E Z
E  Z 
E  Z 
 Z 
E Z
E  Z 
E  Z
Z

E
E ZZ
 E  Z
  Z
E
Es Zs
  Z
 @  @


x12@@ 

x22 @
@
 @  @
 @  @
 @  @
 @  @
 
s @s s @s
@ @

x11 x13 x21 x23

267
 Other examples

The Hamiltonian circuit (HC) and travelling salesman (TS) problems are
NP-complete.

The chromatic number problem is NP-Complete. Given a graph G and a

constant k this problem is to decide whether it is possible to colour the
vertices of the graph with k colours in such a way that each pair of
adjacent (edge connected) vertices are coloured differently.

268
The integer programming problem is NP-compete. An instance of this
problem consists of

1. a set of m pairs (vi, di) in which each vi is a vector of integers of size n

and each di is an integer,

2. a vector d of size n,

3. a constant b.

The problem is to determine if there exists an integer vector x of size n

such that x · vi ≤ di for 1 ≤ i ≤ m and such that x · d ≥ b.

Over the rationals this problem can be solved in polynomial time (linear
programming).

269
The problem of checking the equivalence of nondeterministic finite
automata is NP-hard. Notice that there is no known NP algorithm for
solving this problem. It is complete in the class PSPACE.

270
 8.8 Interpreting NP-completeness

• Worst case analysis. Algorithms that are efficient “on average” are
possible.

• Heuristic methods to limit the exponential number of cases that need

to be examined.

• Approximate solutions for optimisation problems.

• The “usual” instances of problems can satisfy constraints that reduce

to polynomial the complexity of the problem that actually has to be
solved.

271
 8.9 Other complexity classes

The class co-NP is the class of languages L whose complement (Σ∗ − L)

is in NP.

The class EXPTIME is the class of languages decided by a deterministic

Turing machine whose complexity function is bounded by an exponential
function (2p(n) where p(n) is a polynomial).

272
The class PSPACE is the class of languages decided by a deterministic
Turing machine whose space complexity (the number of tape cells used) is
bounded by a polynomial.

The class NPSPACE is the class of languages accepted by a

nondeterministic Turing machine whose space complexity is bounded by a
polynomial.

NP
P⊆ ⊆ PSPACE ⊆ EXPTIME.
co-NP

273