5th Module

• Complexity measures, Polynomial vs Non-Polynomial time

complexity; NP-hard and NP-complete classes, examples.
Reductions
 Complexity definitions
• Big-Oh
• Big-Theta
• Big - Omega
Big Oh (O)

f(n)= O(g(n)) iff there exist positive constants c and n0 such that
f(n) ≤ cg(n) for all n ≥ n0

O-notation to give an upper bound on a function

Omega Notation

Big oh provides an asymptotic upper bound on a function.

Omega provides an asymptotic lower bound on a function.
Theta Notation

Theta notation is used when function f can be bounded both

from above and below by the same function g
How bad is exponential complexity
• Fibonacci example – the recursive fib cannot
even compute fib(50)
 Polynomial vs Non-Polynomial time complexity

Polynomial Non- Polynomial

Problems with solution time bound by a polynomial Problems with solution times not bound by polynomial
of a small degree. (simply non polynomial )
It also called “Tractable Algorithms These are hard or intractable problems
P is the class of problems solved by a deterministic NP is the class of problems which are solved by a non-
Turing machine in a polynomial time deterministic Turing machine in a
Polynomial time, which means it can be
solved by brute force algorithm in an
Exponential time.

Most Searching & Sorting algorithms are polynomial None of the problems in this group has been solved by any
time algorithms polynomial time algorithm
Ex: Ordered Search (O (log n)), Ex: Traveling Sales Person O(n 2 2n)
Polynomial evaluation O(n) Knapsack O(2n/2)
Sorting O(n.log n)
 The class P
• The class P consists of those problems that are
solvable in polynomial time.
• More specifically, they are problems that can
be solved in time O(nk) for some constant k,
where n is the size of the input to the problem
• The key is that n is the size of input
 NP
• NP is not the same as non-polynomial
complexity/running time. NP does not stand
for not polynomial.
• NP = Non-Deterministic polynomial time
• NP means verifiable in polynomial time
• Verifiable?
– If we are somehow given a ‘certificate’ of a
solution we can verify the legitimacy in polynomial
time
There are two classes of non-polynomial time problems
1. NP-Hard
2. NP-Complete

NP-Hard: A problem A is NP-hard if and only if satisfiability

reduces to A (satisfiability α A).
NP Complete Problem: A problem A is NP-complete if and
only if A is NP-hard and A Є NP.
A problem that is NP-Complete can solved in polynomial
time if and only if (iff) all other NP-Complete problems can
also be solved in polynomial time.

All NP-Complete problems are NP-Hard but some NP-Hard

problems are not know to be NP-Complete.
 Review: P and NP
• What do we mean when we say a problem
is in P?
• What do we mean when we say a problem
is in NP?
• What is the relation between P and NP?
 Review: P and NP
• What do we mean when we say a problem
is in P?
– A: A solution can be found in polynomial time
• What do we mean when we say a problem
is in NP?
– A: A solution can be verified in polynomial time
• What is the relation between P and NP?
– A: P  NP, but no one knows whether P = NP
 Review: NP-Complete
• What, intuitively, does it mean if we can
reduce problem P to problem Q?
• How do we reduce P to Q?
• What does it mean if Q is NP-Hard?
• What does it mean if Q is NP-Complete?
 Review: NP-Complete
• What, intuitively, does it mean if we can reduce
problem P to problem Q?
– P is “no harder than” Q
• How do we reduce P to Q?
– Transform instances of P to instances of Q in polynomial
time s.t. Q: “yes” iff P: “yes”
• What does it mean if Q is NP-Hard?
– Every problem PNP p Q
• What does it mean if Q is NP-Complete?
– Q is NP-Hard and Q  NP
 Review:
Proving Problems NP-Complete
• How do we usually prove that a problem R
is NP-Complete?
• A: Show R NP, and reduce a known
NP-Complete problem Q to R
 What happened to automata?
• Problem is in NP iff it is decidable by some non deterministic
Turing machine in polynomial time.
• Remember that the model we have used so far is a
deterministic Turing machine
• It is provable that a Non Deterministic Turing Machine is
equivalent to a Deterministic Turing Machine
• Remember NFA to DFA conversion?
– Given an NFA with n states how many states does the equivalent
DFA have?
– Worst case …. 2n

• The deterministic version of a poly time non deterministic

Turing machine will run in exponential time (worst case)
 Nondeterministic Algorithms:
• Algorithms with the property that the result of every operation
is uniquely defined are termed as deterministic algorithms.
Such algorithms agree with the way programs are executed on
a computer.
• Algorithms which contain operations whose outcomes are not
uniquely defined but are limited to specified set of possibilities.
Such algorithms are called nondeterministic algorithms.
• The machine executing such operations is allowed to choose
any one of these outcomes subject to a termination condition
to be defined later.
• To specify nondeterministic algorithms, there are 3 new
functions.
• Choice(S): arbitrarily chooses one of the elements of sets S
• Failure (): Signals an Unsuccessful completion
• Success (): Signals a successful completion.
Example for Non Deterministic algorithms:
 NP problems
• Graph theory has these fascinating(annoying?)
pairs of problems
– Shortest path algorithms?
– Longest path is NP complete (we’ll define NP
complete later)
– Eulerian tours (visit every vertex but cover every
edge only once, even degree etc). Solvable in
polynomial time!
– Hamiltonian tours (visit every vertex, no vertices
can be repeated). NP complete
 Hamiltonian cycles
• Determining whether a directed graph has a
Hamiltonian cycle does not have a polynomial
time algorithm (yet!)
• However if someone was to give you a
sequence of vertices, determining whether or
not that sequence forms a Hamiltonian cycle
can be done in polynomial time
• Therefore Hamiltonian cycles are in NP
 Satisfiability(SAT)
• A boolean formula is satisfiable if there exists
some assignment of the values 0 and 1 to its variables
that causes it to evaluate to 1.
• I/P : Boolean formula
• O/P : Is formula satisfiable?
SAT Є NP
Proof : Assignment to variables
Verifier: Uses these assignments and checks that the
formula evaluates to true (T).
• CNF – Conjunctive Normal Form. AND ing of clauses of
ORs
 2-CNF SAT
• Each or operation has two arguments that are
either variables or negation of variables
• The problem in 2 CNF SAT is to find
true/false(0 or 1) assignments to the variables
in order to make the entire formula true.

(xy)(yz)(xz)(zy)
(xy)(yz)(xz)(zy)
• Any of the OR clauses can be converted to
implication clauses
 2-SAT is in P
• Create the implication graph

x

y
x

y

z

z
 Satisfiability via path finding
• If there is a path from
• And if there is a path from
• Then FAIL!
• How to find paths in graphs?
– DFS/BFS and modifications thereof
 3 CNF SAT (3 SAT)
• Not so easy anymore.
• Implication graph cannot be constructed
• No known polytime algorithm
• Is it NP?
– If someone gives you a solution how long does it
take to verify it?
– Make one pass through the formula and check
• This is an NP problem
 P is a subset of NP
• Since it takes polynomial time to run the
program, just run the program and get a
solution
• But is NP a subset of P?
• No one knows if P = NP or not
• Solve for a million dollars!
– http://www.claymath.org/millennium-problems
– The Poincare conjecture is solved today
 What is not in NP?
• Undecidable problems
– Given a polynomial with integer coefficients, does it have
integer roots
– Hilbert’s nth problem
– Impossible to check for all the integers
– Even a non-deterministic TM has to have a finite number of
states!
– More on decidability later
• Tautology
– A boolean formula that is true for all possible assignments
– Here just one ‘verifier’ will not work. You have to try all possible
values
 Amusing analogy

• Students believe that every problem assigned

to them is NP-complete in difficulty level, as
they have to find the solutions.
• Teaching Assistants, on the other hand, find
that their job is only as hard as NP, as they
only have to verify the student’s answers.
• When some students confound the TAs, even
verification becomes hard
 Reducibility
• a problem Q can be reduced to another
problem Q’ if any instance of Q can be “easily
rephrased” as an instance of Q’, the solution
to which provides a solution to the instance of
Q
• Is a linear equation reducible to a quadratic
equation?
– Sure! Let coefficient of the square term be 0
 NP - hard
• What are the hardest problems in NP?

• That notation means that L1 is reducible in

polynomial time to L2 .
• The less than symbol basically means that the time
taken to solve L1 is no worse that a polynomial
factor away from the time taken to solve L2.
 NP-hard
• A problem (a language) is said to NP-hard if
every problem in NP can be poly time reduced
to it.
 NP Complete problems/languages
• Need to be in NP
• Need to be in NP-Hard

If both are satisfied then it is an NP complete problem

Reducibility is a transitive relation.

If we know a single problem in NP-Complete that helps when we are asked to prove some
other problem is NP-Complete

Assume problem P is NP Complete

All NP problems are reducible to this problem
Now given a different problem P’
If we show P reducible to P’
Then by transitivity all NP problems are reducible to P’
 What is in NP-Complete
• For this course, we will axiomatically state that
the following problems are NP-Complete
• SAT – Given any boolean formula, is there
some assignment of values to the variables so
that the formula has a true value
• 3-CNF SAT
• Actually any boolean formula can be reduced
to 3-CNF form
 An example reduction
• CLIQUE problem
• A clique in an undirected graph is a subset of
vertices such that each pair is connected by an
edge
• We want to take a problem instance in 3-CNF
SAT and convert it to CLIQUE finding
 Reducing 3CNF SAT to CLIQUE
• Given – A boolean formula in 3 CNF SAT
• Goal – Produce a graph (in polynomial time) such that

• We will construct a graph where satisfying formula

with k clauses is equivalent to finding a k vertex
clique.
 CLIQUE
• An instance of a clique problem gives you 2
things as input
– Graph
– Some positive integer k
• Question being asked = do we have a clique of
size k in this graph
• Why can’t I just go through and pick all
possible k-subsets?
 Decision problems versus optimization
problems
• Finding the maximum sized clique is an
optimization problem
• But we can reduce it to a series of decision
problems
– Can we find a clique of size 3 (why start at 3??)
– Can we find a clique of size 4
– Etc
• In general in our study of NP etc, we will focus
on decision problems
 REDUCE 3-CNF SAT to CLIQUE

For each clause, create a vertex for each literal

For the edges

Connect vertices if they come from different clauses

Even if the vertices come from different clauses, do not connect if it

results in incompatibility. No variable should be connected to its not.

x1 ┐ x1 x1 There are more

edges in here.
Refer to the
CLRS book to get
┐x2 x2 x2 the complete
picture

┐x3 x3 x3
 Vertex cover problem
• A vertex cover of an undirected graph G=(V,E) is a
subset of vertices such that every edge is incident
to at least one of the vertices
• We’re typically interested in finding the minimum
sized vertex cover
• To show vertex cover is NP-complete
• What problem should we try to reduce to it
• It sounds like the ‘reverse’ of CLIQUE
• Reduction is done from CLIQUE to vertex cover
 G’ =
G Complement
Of
G

Clique of size k in G exists iff a vertex cover of size |

V| - k exists in G’ where G’ is the complement graph
(vertices that had an edge between then in G do
not have one in G’ and vice versa)
The original graph has a u,v,x,y CLIQUE. That is a clique of size 4

The complement graph has a vertex cover of size 6 (number of

vertices) – 4 (clique size). z,w is one such vertex cover.
 The reducibility ‘tree’
• Richard Karp proved 21 problems to be NP
complete in a seminal 1971 paper
• Not that hard to read actually!
• Definitely not hard to read it to the point of
knowing what these problems are.
• karp's paper
 Other NP complete problems
• Subset sum
• Given a set of positive integers and some
target t > 0,
do we have a subset that sums up to that
target set
• Why is the naïve algorithm going to be bad?
 Approximation algorithm for Vertex cover

C←∅
while E = ∅
pick any {u, v} ∈ E
C ← C ∪ {u, v}
delete all edges incident to either u or v
return C
 How bad is that approximation?
• Look at the edges returned in that algorithm

• Will the optimum vertex cover include at least one end

point of the edges returned from approx algorithm?