CS 6131

Advanced Algorithm Design and Analysis

MSc in Computer Science

Chapter 7: NP-completeness
Dr. Atef MASMOUDI
Associate Professor
College of Computer Science
[email protected]

Part I – Chapter34: pages : 1343-1677

Introduction to algorithms 1
 What we will cover
• The classes P and NP
• Example of problems to be NP
• NP-completeness
• Reducibility
•NP-hardness

2
 Introduction
• Almost all the studied algorithms have been
polynomial-time algorithms: on inputs of size n,
their worst-case running time is O(nk) for some
constant k.
• You might wonder whether all problems can be
solved in polynomial time. The answer is no.
• Generally, we think of problems that are solvable
by polynomial-time algorithms as being tractable,
or “easy,” and problems that require
superpolynomial time as being intractable, or
“hard.”
3
 Introduction (cont .)
• The subject of this chapter is an interesting class of
problems, called the “NP-complete” problems,
whose status is unknown.
• No polynomial-time algorithm has yet been
discovered for an NP-complete problem, nor has
anyone yet been able to prove that no polynomial-
time algorithm can exist for any one of them.
• This so-called P ≠ NP question has been one of the
deepest, most perplexing open research problems
in theoretical computer science since it was first
posed in 1971.
4
 The classes P and NP
• The class P consists of problems that are solvable in
polynomial time. More specifically, they are problems that
can be solved in O(nk) time for some constant k, where n is
the size of the input to the problem. Most of the problems
examined in previous chapters belong to P.
• The name “NP” stands for “nondeterministic polynomial
time.”
• The class NP consists of those problems that are
“verifiable” in polynomial time.
• Verifiable means If you were somehow given a
“certificate” of a solution, then you could verify that the
certificate is correct in time polynomial in the size of the
input to the problem. 5
 Searching  NP
• Given an array and a target element, does there exist an index such
that the element at that index is equal to the target element?

• Certificate: The certificate is a nondeterministic guess of the index

where the target element is located.

• Verifier: The verifier checks whether the guessed index points to the
target element in the array.

6
 A nondeterministic algorithm for the
searching problem
function NondeterministicLinearSearch(arr, target):
// Nondeterministically guess an index
guessedIndex = choice(indices(arr))
// Verify the guess
if arr [guessedIndex]== target:
// Found the target element at the guessed index
return success()
else:
return failure("Element not found at the guessed index")

7
 Sorting  NP
• The sorting algorithm is designed to arrange elements in a specified
order (e.g., ascending or descending).
• Input:
– An array or list of comparable elements.
– The length of the input array, denoted as n.
• Output:
– A permutation of the input array such that elements are arranged in the
desired order.
• Certificate:
– The certificate in this context is a permutation of the input list that represents
a sorted arrangement according to the target order.
• The verifier can be a simple algorithm that checks that each element
in the permutation is in the correct order according to the target
order. This involves comparing adjacent elements in the
permutation. 8
 A nondeterministic algorithm for the
sorting problem
function NondeterministicSort(arr): function verifyPermutation(permutation,
// Guess a permutation of the input array originalArr):
permutation = nondeterministicGuess(arr) // Check if the guessed permutation is valid
for i from 1 to length(permutation) - 1:
// Verify the guessed permutation if permutation[i] < permutation[i - 1]:
if verifyPermutation(permutation, arr): return failure("Invalid order")
return success(permutation)
else: // Check if the permutation matches the
return failure("Invalid permutation") original array
if permutation is a valid permutation of
originalArr:
function nondeterministicGuess(arr):
// Valid permutation
// Generate a lucky guess for the
permutation return success()
// This could involve nondeterministic else:
choices // Mismatch, reject the permutation
return choice( arr) return failure("Invalid permutation")

9
 Composites  NP
• Composites ={x|x is not prime}
={x| x=yz for integers y,z>1}

• A certificate could be y a factor of x, 1<y<x

• The verifier can be a simple algorithm that checks whether the given
factor is indeed a factor of n. This can be done by dividing x/y and
check for the remainder to be 0

10
 The hamiltonian-cycle problem  NP
• A Hamiltonian cycle (or Hamiltonian circuit) is a cycle that visits each
vertex exactly once.

• Given a directed graph G = (V, E), a certificate would be a sequence

〈v1,v2, v3, …, v|V|〉 of |V| vertices.
• You could check in polynomial time that the sequence contains each
of the |V| vertices exactly once, that (vi, vi+1) ∈ E for i = 1, 2, 3, …,
|V| − 1, and that (v|V|, v1) ∈ E.

11
 SAT  NP
• In logic and computer science, the Boolean satisfiability problem
(abbreviated SATISFIABILITY, SAT or B-SAT) is the problem of
determining if there exists an interpretation that satisfies a given
Boolean formula.
• In other words, it asks whether the variables of a given Boolean
formula can be consistently replaced by the values TRUE or FALSE in
such a way that the formula evaluates to TRUE.
• If this is the case, the formula is called satisfiable.
• On the other hand, if no such assignment exists, the function
expressed by the formula is FALSE for all possible variable
assignments and the formula is unsatisfiable.
• For example, the formula "a AND NOT b" is satisfiable because one
can find the values a = TRUE and b = FALSE, which make (a AND NOT
b) = TRUE. In contrast, "a AND NOT a" is unsatisfiable.
12
 SAT  NP (cont.)
• A Boolean formula is in Conjunctive Normal Form (CNF) if it is the
AND of clauses.
• Each clause is the OR of literals.
• A literal is either a variable or the negation of a variable.
• If a boolean formula is in CNF and every clause consists of exactly k
literals, we say the boolean formula is an instance of k-SAT (k-CNF).
• Example: 3-CNF formula

• For 3-CNF satisfiability, a certificate could be an assignment of

values to variables. You could check in polynomial time that this
assignment satisfies the boolean formula.

13
 NP-complete
• Any problem in P also belongs to NP, since if a problem belongs to P
then it is solvable in polynomial time without even being supplied a
certificate.
• P ⊆ NP. The famous open question is whether P is a proper subset of
NP.
• Informally, a problem belongs to the class NPC—and we call it
NPcomplete—if it belongs to NP and is as “hard” as any problem in
NP.
• We state without proof that if any NP-complete problem can be
solved in polynomial time, then every problem in NP has a
polynomial-time algorithm.

14
 NP-complete (cont.)
• Most theoretical computer scientists believe that the NP-complete
problems are intractable, since given the wide range of NP-complete
problems that have been studied to date—without anyone having
discovered a polynomial-time solution to any of them—it would be
truly astounding if all of them could be solved in polynomial time.
• Given the effort devoted to prove that NP-complete problems are
intractable, we cannot rule out the possibility that the NP-complete
problems could turn out to be solvable in polynomial time.
• As an algorithm designer, if you can establish a problem as NP-
complete, you provide good evidence for its intractability.
• As an engineer, you would then do better to spend your time
developing an approximation algorithm or solving a tractable special
case, rather than searching for a fast algorithm that solves the
problem exactly.
15
 Decision problems VS optimization problems
• Many problems of interest are optimization problems, in which each
feasible (i.e., “legal”) solution has an associated value, and the goal
is to find a feasible solution with the best value.
• For example, in the SHORTEST-PATH problem, the input is an
undirected graph G and vertices u and v, and the goal is to find a
path from u to v that uses the fewest edges.
• In other words, SHORTEST-PATH is the single-pair shortest-path
problem in an unweighted, undirected graph.
• NP-completeness applies directly not to optimization problems,
however, but to decision problems, in which the answer is simply
“yes” or “no” (or, more formally, “1” or “0”).

16
 Decision VS optimization problems (cont.)
• There is usually a way to cast a given optimization problem as a
related decision problem by imposing a bound on the value to be
optimized.
• For example, a decision problem related to SHORTEST-PATH is PATH:
given an undirected graph G, vertices u and v, and an integer k, does
a path exist from u to v consisting of at most k edges?
• The relationship between an optimization problem and its related
decision problem works in our favor when we try to show that the
optimization problem is “hard.”
• That is because the decision problem is in a sense “easier,” or at
least “no harder.”
• As a specific example, you can solve PATH by solving SHORTEST-PATH
and then comparing the number of edges in the shortest path found
to the value of the decision problem parameter k.
17
 NP-completeness and reducibility
• Theoretical computer scientists believe that P ≠
NP comes from the existence of the class of
NPcomplete problems.
• This class has the property that if any NP-
complete problem can be solved in polynomial
time, then every problem in NP has a polynomial-
time solution, that is, P = NP.
• Despite decades of study, no polynomial-time
algorithm has ever been discovered for any NP-
complete problem.
18
 NP-completeness and reducibility (cont.)
• One way that sometimes works for solving a problem is to
recast it as a different problem.
• We call that strategy “reducing” one problem to another.
• Think of a problem Q as being reducible to another problem Q′,
if any instance of Q can be recast as an instance of Q′, and the
solution to the instance of Q′ provides a solution to the instance
of Q.
• Polynomial-time Reduction: A problem A is polynomial-time
reducible to problem B (denoted as ( A ≤p B ) if a polynomial-
time algorithm exists to transform any instance of problem A
into an instance of problem B in such a way that the solution to
the transformed instance of B is the same as the solution to A.

19
 NP-completeness and reducibility (cont.)
• For example, the problem of solving linear equations in an
indeterminate x reduces to the problem of solving quadratic
equations.
• Given a linear-equation instance ax + b = 0 (with solution x =
−b/a), you can transform it to the quadratic equation ax2 + bx +
0 = 0.
• This quadratic equation has the solutions = (−
± − 4 )/2 , where c =0, so that − 4 = . The
solutions are then x = (−b + b)/2a = 0 and x = (−b − b)/2a = −b/a,
thereby providing a solution to ax + b = 0.
• Thus, if a problem Q reduces to another problem Q′, then Q is,
in a sense, “no harder to solve” than Q′.

20
 NP-completeness and reducibility (cont.)
Theorem:
• If any NP-complete problem is polynomial-time
solvable, then P = NP.
• Equivalently, if any problem in NP is not
polynomial-time solvable, then no NP-complete
problem is polynomial-time solvable.

21
 NP-hardness
• A problem is NP-hard if it is at least as hard as the hardest
problems in NP.
• In other words, if you can efficiently solve an NP-hard
problem, you can efficiently solve any problem in NP.
• NP-hardness:
– A decision problem H is NP-hard if every problem in NP is
polynomial-time reducible to H. In simpler terms, if you can solve
an NP-hard problem in polynomial time, you can solve any
problem in NP in polynomial time.
• NP-hard vs. NP-complete:
– NP-complete problems are a subset of NP-hard problems. A
problem is NP-complete if it is both NP-hard and in NP. This
means that NP-complete problems are the hardest problems in
NP, and solving any NP-complete problem in polynomial time
would imply a polynomial-time solution for all problems in NP.

22
NP-hardness (cont.)

23