Approximation Algorithms

Hard Computational Problems

• Some computational problems are hard

• Despite a numerous attempts we do not
know any efficient algorithms for these
problems
• We are also far away from the proof that
these problems are indeed hard to solve, in
other words
• NP=P or NPP, this is a question …
Complexity of Algorithms 2
 Decision/Optimisation Problems
• A decision problem (DP) is a computational problem
for which the intended output is either yes or no

• In an optimisation problem (OP) we rather try to

maximise or minimise some value.

week 10 Complexity of Algorithms 3

 Decision/Optimisation Problems
• An OP can be turned into a DP if we add a parameter
k, and then ask whether the optimal value in OP is at
most or at least k

• Note that if a DP is hard, then its related its

optimisation version must be hard too

week 10 Complexity of Algorithms 4

Decision/Optimisation Problems

• Example:
– Optimisation problem - Given graph G with
integer weights on its edges. What is the weight
of a minimum spanning tree (MST) in G?

– Decision problem – Given graph G with integer

weights on its edges, and an integer k. Does G
have a minimum spanning tree of weight at most
k?
Complexity of Algorithms 5
 The Complexity Class P
• The complexity class P is the set of all decision
problems (or languages) L that can be solved
in worst-case polynomial time.

• That is, there is an algorithm A that if x  L,

then on input x, A outputs yes in time p(n),
where n is the size (length) of x and p(n) is a
polynomial

week 10 Complexity of Algorithms 6

 In P or not in P: Examples
• In P:
– Shortest path
– Minimum Spanning Tree

• Not in P (NP):
– Vertex Cover
– Traveling salesman

My T. Thai
7
[email protected]
 Vertex-Cover is in NP

• Suppose we are given an integer k and a graph G

• The non-deterministic algorithm chooses a subset
of vertices C  V, s.t., ¦C¦  k, and then it checks
deterministically whether this subset C is an
appropriate vertex cover

week 10 Complexity of Algorithms 8

 Vertex Cover

• Given a graph G=(V,E),

– a vertex cover for G is a subset CV, s.t.,
– for every edge (v,w) in E, vC or wC

• The optimisation problem is to find as small a vertex

cover as possible

• Vertex-Cover is the decision problem that takes a graph

G and an integer k as input, and asks whether there is a
vertex cover for G containing at most k vertices
week 10 Complexity of Algorithms 9
 What is “hard”
• Decision Problems: Only consider YES-NO
– Decision version of TSP: Given n cities, is there a TSP
tour of length at most L?

• Why Decision Problem? What relation between

the optimization problem and its decision?
– Decision is not “harder” than that of its optimization
problem
– If the decision is “hard”, then the optimization problem
should be “hard”
My T. Thai
10
[email protected]
 Why Approximation Algorithms
• Approximation algorithms are used for the
Traveling Salesman Problem (TSP) because finding
an exact, optimal solution is computationally
infeasible for large instances of the problem. TSP is
an NP-hard problem, meaning no known algorithm
can find an optimal solution in polynomial
time. Approximation algorithms offer quick, near-
optimal solutions, even for large datasets,
providing a practical alternative to searching for
the absolute shortest route.
 Why Approximation Algorithms are Needed:

• Computational Complexity:
• The number of possible routes grows exponentially with the number
of cities, making finding the absolute shortest route impractical for
even moderately sized problems.

• No Known Polynomial-Time Solution:

• Computer scientists have not yet found an efficient algorithm (running
in polynomial time) to solve TSP optimally.

• Practicality:
• In real-world scenarios, finding a route that is within a certain
percentage of the optimal solution can be much faster and more
useful than attempting to find the absolute shortest route.
 Approximation Algorithms
• An algorithm that returns near-optimal solutions in
polynomial time

• Approximation Ratio ρ(n):

– Define: C* as a optimal solution and C is the solution produced by
the approximation algorithm
– max (C/C*, C*/C) <= ρ(n)

– Maximization problem: 0 < C <= C*, thus C*/C shows that C* is

larger than C by ρ(n)
– Minimization problem: 0 < C* <= C, thus C/C* shows that C is
larger than C* by ρ(n)
My T. Thai
13
[email protected]
 A Dilemma!
• We cannot find C*, how can we compare C to
C*?
• How can we design an algorithm so that we
can compare C to C*

My T. Thai
14
[email protected]
 Approximation Schemes
• Let c(S) be the value of a solution S delivered by an
algorithm A to an optimisation problem P and OPT
will be the optimal solution for P
• We say that A is a -approximation algorithm for a
minimisation problem P if
– c(S)  ·OPT
• And A is is a -approximation algorithm for a
maximisation problem P if
– c(S)  ·OPT

week 10 Complexity of Algorithms 15

 Approximation Schemes
• If C(S):OPT = 1, it means the solution S is
exactly as good as the optimal solution.
• If C(S):OPT > 1, it means the solution S is
worse than the optimal solution (the
algorithm A is not perfectly accurate).
• If C(S):OPT < 1, it means the solution S is
better than the optimal solution (this can't
happen in a true optimization problem, as OPT
is defined as the best possible
week 10 Complexity of Algorithms 16
 PTAS (Polynomial Time Approximation
Scheme):
• A PTAS (Polynomial Time Approximation Scheme) is an
algorithm that provides a solution within a factor of 1 + ε of
the optimal solution for an optimization problem, where ε is a
parameter controlling the accuracy. Critically, the algorithm's
runtime is polynomial in the input size for any fixed ε > 0,
meaning the runtime complexity is polynomial in the input
size and the parameter ε does not affect the polynomial
nature of the runtime.

My T. Thai
17
[email protected]
 PTAS (Polynomial Time Approximation
Scheme):
• Approximation Guarantee:
• A PTAS guarantees a solution that is within a factor
of 1 + ε of the optimal solution. For example, if ε =
0.1, the solution will be within 110% of the optimal
solution.
• Polynomial Time:
• The runtime of a PTAS is polynomial in the input size
(n) for any fixed ε. This means the time complexity
can be expressed as something like O(n^k) for some
constant k, where n is the input size.
 PTAS (Polynomial Time Approximation
Scheme):
• Parameterization:
• A PTAS takes an additional parameter ε, which
controls the accuracy of the approximation. A
smaller ε results in a more accurate solution,
but the runtime may increase.
 Fully Polynomial-Time Approximation
Scheme (FPTAS)
• FPTAS is an algorithm that finds approximate
solutions to optimization problems, especially
those that are NP-hard. It guarantees a
solution within a factor of (1 + ε) of the
optimal solution, where ε is a small positive
value. The key characteristic of an FPTAS is
that its running time is polynomial in both the
size of the input and 1/ε.
 FPTAS
• Approximation Guarantee:
• FPTAS algorithms provide solutions that are
close to the optimal solution, with the
closeness controlled by the parameter ε.
• Polynomial Running Time:
• The time complexity of an FPTAS is bounded by
a polynomial function of the input size (n) and
1/ε, making it efficient for large problems.
• Dynamic Programming:
• Many FPTAS algorithms are based on dynamic programming, which
allows them to find optimal solutions in a structured manner, even if
the time complexity is not always polynomial.

• NP-hard problems:
• FPTAS algorithms are particularly useful for NP-hard problems, where
finding exact solutions is computationally expensive.

• Time complexity:
• The time complexity of an FPTAS is generally expressed as O(poly(n,
1/ε)), where poly(n, 1/ε) represents a polynomial function of the input
size (n) and the inverse of the desired precision (1/ε).
 Relationship of PTAS to FPTAS:

• A PTAS is a weaker notion than a Fully Polynomial-Time

Approximation Scheme (FPTAS). While both algorithms offer
approximation guarantees, an FPTAS requires that the runtime
is also polynomial in 1/ε, whereas a PTAS only requires
polynomial runtime for fixed ε.

• In simpler terms: A PTAS allows you to get a good

approximation of a solution to a problem within a certain
accuracy (1 + ε), and it does this relatively quickly, with a
runtime that's polynomial in the input size. However, the
runtime might become slower if you try to make the
approximation very accurate (small ε).
 Vertex Cover
• Definition:
Given an undirected graph G (V , E ), a subset V '  V is called
a vertex cover of G iff for every edge e  E , e has at least
one endpoint incident at V '

• An Example

My T. Thai
25
[email protected]
 Vertex Cover Problem
• Definition:
– Given an undirected graph G=(V,E), find a vertex
cover with minimum size (has the least number of
vertices)
– This is sometimes called cardinality vertex cover

My T. Thai
26
[email protected]
 How to solve it
• Matching:
– A set M of edges in a graph G is called a matching
of G if no two edges in set M have an endpoint in
common
• Example:

My T. Thai
27
[email protected]
 How to solve it (cont)
• Maximum Matching:
– A matching of G with the greatest number of edges
• Maximal Matching:
– A matching which is not contained in any larger matching
• Note: Any maximum matching is certainly maximal,
but not the reverse.

My T. Thai
28
[email protected]
 Main Observation
• No vertex can cover two edges of a matching
• The size of every vertex cover is at least the size of every
matching: |M| ≤ |C|
• |M| = |C| indicates the optimality

• Possible Solution: Using Maximal matching to find Minimum

vertex cover
My T. Thai
29
[email protected]
 An Algorithm
• Alg 1: Find a maximal matching in G and output the set
of matched vertices
• Theorem: The algorithm 1 is a factor 2-approximation
algorithm.
• Proof:
Let opt be a size of an optimal solution. Let C be a set of
vertex cover obtained from algorithm 1. We have :
| M |opt and | C |2 | M |
Therefore, | C |2opt

My T. Thai
30
[email protected]
 My T. Thai
31
[email protected]
32
 My T. Thai
33
[email protected]
 Traveling-salesman Problem

• In the traveling salesman Problem, a salesman must

visits n cities. We can say that salesman wishes to
make a tour or Hamiltonian cycle, visiting each city
exactly once and finishing at the city he starts from.

• There is a non-negative cost c (i, j) to travel from

the city i to city j. The goal is to find a tour of
minimum cost. We assume that every two cities are
connected. Such problems are called Traveling-
salesman problem (TSP).
My T. Thai
34
[email protected]
 Hamiltonian Cycle is NP
• Hamiltonian-Cycle is the problem that takes a graph
G as an input and asks whether there is a simple
(Hamiltonian) cycle in G that visits every vertex of G
exactly once

• The non-deterministic algorithm chooses a cycle and

then it checks deterministically whether this cycle is
indeed a Hamiltonian cycle in G

week 10 Complexity of Algorithms 35

 Traveling-salesman Problem

• We can model the cities as a complete graph

of n vertices, where each vertex represents a
city.
• It can be shown that TSP is NPC.
• If we assume the cost function c satisfies the
triangle inequality, then we can use the
following approximate algorithm.

My T. Thai
36
[email protected]
 My T. Thai
37
[email protected]
38
• Intuitively, Approx-TSP first makes a full walk
of MST T, which visits each edge exactly two
times.
• To create a Hamiltonian cycle from the full
walk, it bypasses some vertices (which
corresponds to making a shortcut)

My T. Thai
39
[email protected]
 TSP starting point
• What would be a good baseline?
• Something we can calculate that would at
least connect things up for us.
• A Minimum Spanning Tree!
A 3
4
5 5
B 6 D

4 3 3
4
C E
2
 From MST to Tour
A 3
• How do we get from start to every vertex 4
and back? 5 5
B 6 D
• Make the starting point the root, do a
4 3 3
traversal (DFS) of the graph! 4
C E
• Why not BFS? Because the “next vertex” 2
isn’t always right next to you! (not a If is
problem in this example, but very bad if Go f
To D
you have a very tall tree)
Back

• How much gas do we use in DFS? We use

each edge twice
 Doing a Little Better
• Using each edge twice is potentially a little wasteful. Can we do better?

• The biggest problem is vertices of odd degree. The last time we enter that
vertex, the only way out is an already used edge.
• And that’s definitely not taking us somewhere new!

• So lets add some possible ways out. A 3

4
5 5
B 6 D

4 3 3
4
C E
2
 A
What would help? 4
5 5
3

• A matching would help! (i.e. a set of edges B 6 D

that don’t share endpoints)
4 3 3
• Specifically a minimum weight matching. 4
C E
• You can find one of those efficiently (just 2
trust me)
• Add those edges in (if they’re already in A 3
4
the MST, make an extra copy)! 5 5
B 6 D
• So we now have the MST AND the
minimum weight matching on the odd 4 3 3
4
edges. ( odd degree, in and out requires
degree 2) C E
2
 A

Did It Help?
B D
• So…now every vertex has even degree…but
there’s not a nice order anymore.
• We’ll have to find one. C E

• Start from the starting point, and just follow A

any unused edge!
• Because every vertex has even degree, B D
(except for the starting vertex) when you go
in, you can come out! So you can’t “get stuck”
• What if you get back to the start and end up
C E
with unused edges? Find a visited vertex one
is adjacent to and “splice in” the cycles.
D,A,B,E,D is found first. E,C,E found
next. After splicing:
D,A,B,E,C,E,D. is the final tour
 Is it a good approximation algorithm?
• We will visit every vertex at least once!
• Every vertex had degree at least one (because
we started with an MST!)
• So by the end of the process, we had degree
at least two on every vertex.
• And we go back and use all the edges we
selected. So we visit every vertex, and we start
and end at the same place.
 Christofides' algorithm

• The basic strategy of the double-tree algorithm is to construct an

Eulerian tour whose total cost is at most α,OPT , then shortcut it
to get an α-approximation solution. The same strategy can be
carried out to yield a 3/2-approximation algorithm.

• Christofides' algorithm
– Find a minimum spanning tree T
– Let O be the set of nodes with odd degree in T
– Find a minimum-cost perfect matching M on O
– Add the set of edges of M to T
– Find an Eulerian tour.
– Shortcut the Eulerian tour.
 Graph Coloring Problem
• Approximation algorithms for graph
coloring aim to find a coloring of a graph with
a number of colors close to the chromatic
number, which is the minimum number of
colors needed to color the graph. These
algorithms are useful because finding the
exact chromatic number is NP-hard.
• Yes, the graph coloring algorithm is NP-
complete for k greater than 3, meaning it's a
problem where finding a solution is
computationally difficult, and verifying a
solution is easy.
• Specifically, determining if a graph can be
colored with k colors is NP-complete k greater
than 3, while the problem is easily solvable for
k=2
 Greedy Algorithm:

– Concept:
– Assign colors to vertices one by one, ensuring that
no adjacent vertices have the same color.
– Process:
• Start with an arbitrary ordering of vertices.
• For each vertex, try to assign the smallest possible color
that isn't used by any of its neighbors.
– Performance:
• While simple and fast, the greedy algorithm doesn't
guarantee a coloring close to the optimal chromatic
number.
 Maximal Independent Set
• Independent Set: A set of vertices in a graph
where no two vertices are connected by an
edge.
• Maximal Independent Set: An independent
set that cannot be made larger by adding any
additional vertices.
• Maximum Independent Set: An independent
set of the largest possible size in the graph.
 Chromatic Number
• Consider a k-chromatic graph G of n vertices
properly colored with k different colors. Since
the largest number of vertices in G with the
same color cannot exceed the independence
number β(G) >= n/k.
• β(G) – largest set size
• Find minimal number of the maximal
independent sets which collectively include all
vertices of G.
 Independent Sets for this graph

• {a,c,d,f} - maximal ( also maximum )

• {b,g} - maximal
• {a,c,d,g} - maximal ( also maximum)
• {b,f} - maximal
• {a,e} - maximal
 Chromatic Number
• Find the minimal number of maximal
independent sets, which collectively include all
vertices of G.
• {a,c,d,f} - maximal ( also maximum )
• {b,g} - maximal
• {a,c,d,g} - maximal ( also maximum)
• {b,f} - maximal
• {a,e} - maximal

• {a,c,d,f} +{b,g} + {a,e}

• {a,c,d,g} + {b,f} + {a,e}
• Chromatic number if therefore 3.
• Largest size (4) >= n(7)/k(3) >= 2.3
 Probabilistically good algorithms
• Randomized algorithms, use randomness to solve
problems or approximate solutions with a certain
probability of accuracy.

• Unlike deterministic algorithms, they can produce

different outputs for the same input, depending
on the random choices made during execution.
• They can be used for tasks like factoring, and
finding Hamiltonian circuits.
 Key Features of Probabilistic Algorithms:

• Randomness: They incorporate random elements, often using a

random number generator.

• Not Deterministic: They don't always produce the same output

for the same input.

• Probability of Success: They aim to solve problems or

approximate solutions with a high probability of correctness or
accuracy.

• Efficiency: They can sometimes offer speedup compared to

deterministic algorithms for certain problems.
 Types of Probabilistic Algorithms:

• Monte Carlo Algorithms:

– It is a probabilistic algorithm that has a certain
probability of returning the correct answer whatever
input is considered.

• Las Vegas Algorithms:

– These algorithms always produce correct results,
never returns an incorrect answer , but they might
return an answer at all (running time may vary
depending on the input).
 Monte Carlo Approximation
• 0<p<1, a p-correct Monte Carlo algorithm is a
probabilistic algorithm that reduces the
correct answer with probability not less than
p, no matter what input is considered.
• Many times there is no efficient method
available to test whether an answer returned
by Monte Carlo algorithm for a given input is
correct.
 Las Vegas Approximation
• Could run arbitrarily long without producing an
answer.
• It could make a random decision from which there
is no recovery (a dead end)
• After disaster it terminates and restarts.
• The probability of reaching a dead end on any given
run is usually quite small, thus running the
algorithm several times significantly increases the
probability that the algorithm successfully solves
the problem.
 Las Vegas Approximation
• MC Algorithm for which there is no efficient
way to test whether the returned result is
correct can be turned into a LV Algorithm by
simply repeating the algorithm in case of
receiving an incorrect answer.