NP Completeness

Decision problem/ Decision algorithm: Any problem for which the answer is either zero or
one is decision problem. Any algorithm for a decision problem is termed a decision algorithm.

Optimization problem/ Optimization algorithm: Any problem that involves the identification
of an optimal (either minimum or maximum) value of a given cost function is known as an
optimization problem. An optimization algorithm is used to solve an optimization problem.

Nondeterministic Algorithms

• Algorithms with the property that the result of every operation is uniquely defined are
termed as deterministic algorithms. Such algorithms agree with the way programs are
executed on a computer.
• Algorithms which contain operations whose outcomes are not uniquely defined but are
limited to specified set of possibilities. Such algorithms are called non-deterministic
algorithms.
• The machine executing such operations is allowed to choose any one of these outcomes
subject to a termination condition to be defined later.

To specify nondeterministic algorithms, there are 3 new functions.

Choice(S)→ arbitrarily chooses one of the elements of sets S

Failure ()→ Signals an Unsuccessful completion
Success ()→ Signals a successful completion.

Example for non-deterministic algorithms:

Algorithm Search(x){ Whenever there is a set of choices

//Problem is to search an element x that leads to a successful completion
//output J, such that A[J]=x; or J=0 if x is not in A then one such set of choices is always
J:=Choice(1,n);
made and the algorithm terminates.
if( A[J]:=x) then {
Write(J);
Success (); A Nondeterministic algorithm
} terminates unsuccessfully if and only
else{ if (iff) there exists no set of choices
write (0); leading to a successful signal.
failure ();
}

CS313 DESIGN AND ANALYSIS OF ALGORITHMS 1

 Nondeterministic Knapsack algorithm

Algorithm DKP(p, w, n, m, r, x){ p→ given Profits

W:=0; w→ given Weights
P:=0; n→ Number of elements
for i:=1 to n do{ m→ Maximum capacity of bag
x[i]:=choice(0, 1); P→Final Profit
W:=W+x[i]*w[i]; W→Final weight
P:=P+x[i]*p[i]; r → Min profit / Threshold on profit
}
if( (W>m) or (P<r) ) then Failure();
else Success();
}

The Classes of Problems

P Class Problems

• Problems that are in “P” class can be solved in O(Nc) (where c is some constant) by a
deterministic polynomial time algorithm.
• P is often a class of computational problems that are solvable and tractable. Tractable
means that the problems can be solved in theory as well as in practice. But the problems
that can be solved in theory but not in practice are known as intractable.
• We say we can solve these in “polynomial time”. These are the easy problems.

➢ Binary Search
➢ Sorting
➢ Matrix Multiplication
➢ Shortest Path Problems
➢ Minimal Spanning Tree
➢ Etc…

NP Class Problems

Collection of decision problems that can be solved by a non-deterministic algorithm in polynomial

time. Problems are of “NP class” if it is easy to check the correctness of a claimed solution. In other
words, the solution can be checked in polynomial time.

• This doesn’t say it is easy to find a solution. In fact, it is often hard to find a
solution!

• Example problems include the Traveling Salesman Problem and the Hamiltonian
Circuit Problem. Note it is easy to verify a solution.

NP problems are again classified into 2 types. I) NP Hard II) NP Complete

CS313 DESIGN AND ANALYSIS OF ALGORITHMS 2

NP Hard Problems

• NP-hard problems are those that are at least as hard as the hardest problems in NP. NP-
hard problems are not required to have a solution verifiable in polynomial time.
• The concept of NP-hardness captures problems that are, in a sense, "as difficult as" NP-
complete problems, but they may not have a yes/no answer (making them a
generalization of NP-complete problems).

Key Characteristics of NP-Hard Problems:

1. At least as hard as NP-complete problems: If an NP-hard problem could be solved in

polynomial time, then every problem in NP could also be solved in polynomial time. This
would imply that P = NP.

2. May not have polynomial-time verifiable solutions: Unlike NP-complete problems, NP-
hard problems are not required to have solutions that can be verified in polynomial time.

3. May not be decision problems: NP-hard problems can be more general than decision
problems (yes/no problems). Many NP-hard problems are optimization problems.

Formal Definition for NP-Hard Problem

A problem A is NP-hard if every problem L in NP can be reduced to A in polynomial time. This

means that any NP problem can be translated into an instance of A, such that solving A would
solve L as well.

Examples of NP-Hard Problems

1. Travelling Salesperson Problem (TSP) (Optimization version):

o Problem: Given a list of cities and the distances between them, find the shortest
possible route that visits each city exactly once and returns to the starting city.
o Why NP-Hard: The optimization version of TSP (finding the shortest path) is NP-
hard because it requires searching through many possible paths to find the
optimal solution, and it is computationally infeasible to check all possible
combinations in a reasonable time for large inputs.
o Decision version (which asks if a path exists with a total cost less than a given
value) is NP-complete, but the optimization version is NP-hard.

2. Knapsack Problem (Optimization version):

o Problem: Given a set of items, each with a weight and a value, determine the
number of each item to include in a collection such that the total weight does not
exceed a limit and the total value is maximized.
o Why NP-Hard: Finding the exact best combination of items is NP-hard because
there are an exponential number of possible ways to fill the knapsack. The
decision version (whether it’s possible to reach a certain value with given
constraints) is NP-complete, but optimizing for the maximum value makes it NP-
hard.

CS313 DESIGN AND ANALYSIS OF ALGORITHMS 3

NP Complete Problems

NP-complete problems are a class of problems that are both in NP and as hard as any problem
in NP. These problems represent the intersection of the classes NP (nondeterministic polynomial
time) and NP-Hard.

Key Characteristics of NP-Complete Problems:

1. In NP: NP-complete problems belong to the class NP, meaning their solutions can be
verified in polynomial time. If a proposed solution is given, it can be checked efficiently
(in polynomial time) whether the solution is correct.
2. NP-Hard: NP-complete problems are at least as hard as any problem in NP. This means
that any problem in NP can be transformed (or reduced) to an NP-complete problem in
polynomial time. Solving an NP-complete problem efficiently (in polynomial time) would
imply that every problem in NP can be solved efficiently, thus showing that P = NP.
3. Equivalence in Hardness: All NP-complete problems are inter-reducible, meaning that
if one NP-complete problem can be solved in polynomial time, then every NP-complete
problem can be solved in polynomial time.

Why Are NP-Complete Problems Important?

• NP-complete problems are central to computer science because they represent the
boundary between problems that we know how to solve efficiently (in polynomial time)
and problems that we don’t know how to solve efficiently. If any NP-complete problem is
solved efficiently, then all problems in NP can also be solved efficiently, effectively proving
that P = NP.
• However, no polynomial-time algorithm has yet been discovered for any NP-complete
problem, and it remains one of the greatest open questions in theoretical computer
science.

Examples of NP-Complete Problems

1. Satisfiability Problem (SAT):

o Problem: Given a Boolean formula (a logical expression using AND, OR, and NOT
operators), is there an assignment of true/false values to variables such that the
formula evaluates to true?
o Why NP-Complete: The SAT problem was the first problem proven to be NP-
complete (Cook’s theorem). It is in NP because once we have a satisfying
assignment of variables, we can quickly verify that the formula is true. It is NP-
hard because any NP problem can be reduced to SAT in polynomial time.
2. Hamiltonian Cycle Problem:
o Problem: Given a graph, does there exist a cycle that visits each vertex exactly
once and returns to the starting point (a Hamiltonian cycle)?
o Why NP-Complete: Verifying whether a given cycle is Hamiltonian is easy (in
NP), but finding such a cycle is as hard as any NP problem (NP-hard).

CS313 DESIGN AND ANALYSIS OF ALGORITHMS 4

Polynomial Time Verification

Polynomial time verification is a key concept for understanding the class NP (Nondeterministic
Polynomial time). This concept relates to how quickly a solution to a problem can be verified,
even if the process of finding the solution may be more complex.

Relationship between different classes of problems

What is Verification?

Verification involves checking whether a given solution to a problem is correct. For many
problems, we may not know how to quickly find a solution, but if a solution is provided, we can
efficiently verify whether it is correct.

Polynomial Time

A verification process is said to be in polynomial time if the time it takes to verify the solution
grows at most as a polynomial function of the size of the input. This means that as the size of the
input (denoted by n) increases, the time taken to verify the solution is bounded by a function like
n², n³, etc., rather than an exponential or factorial function like 2ⁿ or n!. Problems that can be
verified in polynomial time are part of the NP class.

Formal Definition

Let’s define this more formally:

• A decision problem (one that has a yes/no answer) is in NP if:

1. There is a solution (also called a "certificate" or "witness") to the problem.

2. Given this solution, there is a verification algorithm that can check if the solution
is correct in polynomial time.

Example: PATH Problem

• The problem denotes the decision problem version of shortest path.

• Given a graph G, a start vertex u, and an end vertex v. Does there exist a path in G, from u
to v of length at most k?
• The instance is: G = ({A, B, C, D}, u=A, v=B, {(A, C,2), (A, D, 15), (C, D, 3), (D, B, 1)} k=6
• A certificate y = (A, C, D, B) (Certificate means possible solution)

CS313 DESIGN AND ANALYSIS OF ALGORITHMS 5

 • This certificate can be verified in polynomial time.

Example: Traveling Salesperson Problem (TSP):

• The problem asks whether there is a route that visits all cities exactly once with a total
distance less than some value D.

• While it is hard to find the best route among many cities (this process might take
exponential time), if someone gives you a proposed route, you can easily verify if the total
distance is less than D in polynomial time by summing up the distances.

Example: Boolean Satisfiability Problem (SAT)

The SAT problem asks whether there exists an assignment of true/false values to variables that
satisfies a given Boolean formula.

• Finding this satisfying assignment may be hard (we don't know a quick way to solve it),
but if someone provides an assignment, we can verify whether it satisfies the formula in
polynomial time by simply evaluating the formula.

Importance of Polynomial Time Verification

Polynomial time verification is central to the definition of NP because it means that for problems
in NP, even if we cannot solve them efficiently (in polynomial time), we can still verify the
solutions efficiently. The class NP stands for "Nondeterministic Polynomial time" because, in
theory, if we had a nondeterministic computer, it could guess the solution and verify it in
polynomial time.

Reducibility

Reducibility is a key concept used to compare the difficulty of different computational problems.
If one problem can be "reduced" to another, it means that the first problem can be solved by
transforming it into the second problem. This transformation, or reduction, helps to understand
the relative complexity of problems and is fundamental to classifying problems as NP-complete
or NP-hard.

Definition of Reducibility

• A problem A is said to be reducible to a problem B (written as A ≤p B) if there is an

efficient way (i.e., a polynomial-time algorithm) to transform instances of A into instances
of B such that solving B allows us to solve A.
• If problem A can be reduced to problem B, and B is easier or equally difficult as A, solving
B automatically gives us a way to solve A. Therefore, reducibility helps in determining
how difficult it is to solve a problem by relating it to other known problems.

CS313 DESIGN AND ANALYSIS OF ALGORITHMS 6

 ➢ Classifying Problems: Reducibility helps classify problems into complexity classes,
like NP-complete, NP-hard, or P. For instance, if a problem is shown to be NP-
complete, it means every other problem in NP can be reduced to it in polynomial time,
making it as hard as any problem in NP.

➢ Proving NP-completeness: The concept of reducibility is essential in proving that a

problem is NP-complete. If a known NP-complete problem can be reduced to a new
problem, and the new problem is in NP, then the new problem is also NP-complete.
This is done using a technique called reduction to show that one problem is at least
as hard as another NP-complete problem.

Examples of Reducibility

1. Reducing SAT to 3-SAT:

o The Boolean satisfiability problem (SAT) was the first problem proven to be
NP-complete. The 3-SAT problem is a special case of SAT where each clause has
exactly three literals.

o Reduction: Every instance of SAT can be transformed into an instance of 3-SAT

using a polynomial-time algorithm. This shows that 3-SAT is at least as hard as
SAT, and since SAT is NP-complete, 3-SAT is also NP-complete.

2. Reducing 3-SAT to CLIQUE:

o The CLIQUE problem asks whether a graph contains a complete subgraph (where
every pair of vertices in the subgraph is connected) of a given size.

o Reduction: An instance of 3-SAT can be transformed into an instance of CLIQUE

in polynomial time, which shows that solving CLIQUE is at least as hard as solving
3-SAT. Since 3-SAT is NP-complete, CLIQUE is also NP-complete.

Polynomial-time Reductions and NP-Completeness

To prove that a problem is NP-complete, two things must be shown:

1. Membership in NP: The problem must belong to the class NP, meaning that given a
solution, it can be verified in polynomial time.

2. Polynomial-time Reduction: There must be a known NP-complete problem that can be

reduced to the given problem in polynomial time. This proves that the problem is at least
as hard as any NP-complete problem.

For example, to prove that the CLIQUE problem is NP-complete, we can take an NP-complete
problem like 3-SAT and show that every instance of 3-SAT can be transformed into an instance of
CLIQUE in polynomial time.

CS313 DESIGN AND ANALYSIS OF ALGORITHMS 7