Computability and decidability

Computable Problems –
You are familiar with many problems (or functions) that are computable (or decidable), meaning
there exists some algorithm that computes an answer (or output) to any instance of the problem
(or for any input to the function) in a finite number of simple steps.A simple example is the
integer increment operation:

f(x) = x + 1

It should be intuitive that given any integer x, we can compute x + 1 in a finite number of steps.
Since x is finite, it may be represented by a finite string of digits. Using the addition method (or
algorithm) we all learned in school, we can clearly compute another string of digits representing
the integer equivalent to x + 1.
Yet there are also problems and functions that that are non-computable (or undecidable or
uncomputable), meaning that there exists no algorithm that can compute an answer or output
for all inputs in a finite number of simple steps. (Undecidable simply means non-computable in
the context of a decision problem, whose answer (or output) is either “true” or “false”).

Non-Computable Problems –
A non-computable is a problem for which there is no algorithm that can be used to solve it. Most
famous example of a non-computablity (or undecidability) is the Halting Problem. Given a
description of a Turing machine and its initial input, determine whether the program, when
executed on this input, ever halts (completes).

The alternative is that it runs forever without halting. The halting problem is about seeing if a
machine will ever come to a halt when a certain input is given to it or if it will finish running. This
input itself can be something that keeps calling itself forever which means that it will cause the
program to run forever.

Other example of an uncomputable problem is: determining whether a computer program loops
forever on some input. You can replace “computer program” by “Turing machine or algorithm”if
you know about Turing machine.

Proving Computability or Non-Computability –

We can show that a problem is computable by describing a procedure and proving that the
procedure always terminates and always produces the correct answer. It is enough to provide a
convincing argument that such a procedure exists finding the actual procedure is not necessary
(but often helps to make the argument more convincing).

To show that a problem is not computable, we need to show that no algorithm exists that solves
the problem. Since there are an infinite number of possible procedures, we cannot just list all
possible procedures and show why each one does not solve the problem. Instead, we need to
construct an argument showing that if there were such an algorithm it would lead to a
contradiction

The core of our argument is based on knowing the Halting Problem is noncom- putable. If a
solution to some new problem P could be used to solve the Halting Problem, then we know that
P is also noncomputable. That is, no algorithm exists that can solve P since if such an algorithm
exists it could be used to also solve the Halting Problem which we already know is
impossible.The proof technique where we show that a solution for some problem P can be used
to solve a different problem Q is known as a reduction. A problem Q is reducible to a problem
P if a solution to P could be used to solve Q. This means that problem Q is no harder than
problem P, since a solution to problem Q leads to a solution to problem P.

Turing Machine Halting Problem

Input − A Turing machine and an input string w.

Problem − Does the Turing machine finish computing the string w in a finite number of steps?
The answer must be either yes or no.

Proof − At first, we will assume that such a Turing machine exists to solve this problem and
then we will show it is contradicting itself. We will call this Turing machine as a Halting machine
that produces a ‘yes’ or ‘no’ in a finite amount of time. If the halting machine finishes in a finite
amount of time, the output comes as ‘yes’, otherwise as ‘no’. The following is the block diagram
of a Halting machine −

Now we will design an inverted halting machine (HM)’ as −

● If H returns YES, then loop forever.

● If H returns NO, then halt.

The following is the block diagram of an ‘Inverted halting machine’ −

Further, a machine (HM)2 which input itself is constructed as follows −

● If (HM)2 halts on input, loop forever.

● Else, halt.

Here, we have got a contradiction. Hence, the halting problem is undecidable.

Reduction

Reduction is a technique in which if a problem P1 is reduced to a problem P2 then any solution

of P2 solves P1. In general, if we have an algorithm to convert an instance of a problem P1 to
an instance of a problem P2 that have the same answer then it is called as P1 reduced P2.
Hence if P1 is not recursive then P2 is also not recursive. Similarly, if P1 is not recursively
enumerable then P2 also is not recursively enumerable.

Theorem: if P1 is reduced to P2 then

1. If P1 is undecidable, then P2 is also undecidable.

2. If P1 is non-RE, then P2 is also non-RE
Proof:

1.Consider an instance w of P1. Then construct an algorithm such that the algorithm takes
instance w as input and converts it into another instance x of P2. Then apply that algorithm to
check whether x is in P2. If the algorithm answer 'yes' then that means x is in P2, similarly we
can also say that w is in P1. Since we have obtained P2 after reduction of P1. Similarly if
algorithm answer 'no' then x is not in P2, that also means w is not in P1. This proves that if P1 is
undecidable, then P1 is also undecidable.

2.We assume that P1 is non-RE but P2 is RE. Now construct an algorithm to reduce P1 to P2,
but by this algorithm, P2 will be recognized. That means there will be a Turing machine that
says 'yes' if the input is P2 but may or may not halt for the input which is not in P2. As we know
that one can convert an instance of w in P1 to an instance x in P2. Then apply a TM to check
whether x is in P2. If x is accepted that also means w is accepted. This procedure describes a
TM whose language is P1 if w is in P1 then x is also in P2 and if w is not in P1 then x is also not
in P2. This proves that if P1 is non-RE then P2 is also non-RE
Post Correspondence Problem

The Post Correspondence Problem (PCP), introduced by Emil Post in 1946, is an undecidable
decision problem. The PCP problem over an alphabet ∑ is stated as follows −

Given the following two lists, M and N of non-empty strings over ∑ −

M = (x1, x2, x3,………, xn)

N = (y1, y2, y3,………, yn)

We can say that there is a Post Correspondence Solution, if for some i1,i2,………… ik, where 1
≤ ij ≤ n, the condition xi1 …….xik = yi1 …….yik satisfies.

Example 1

Find whether the lists

M = (abb, aa, aaa) and N = (bba, aaa, aa)

have a Post Correspondence Solution?

Here,

x2x1x3 = ‘aaabbaaa’

and y2y1y3 = ‘aaabbaaa’

We can see that

x2x1x3 = y2y1y3

Hence, the solution is i = 2, j = 1, and k = 3.

Example 2

Find whether the lists M = (ab, bab, bbaaa) and N = (a, ba, bab) have a Post Correspondence
Solution?

Solution

In this case, there is no solution because −

| x2x1x3 | ≠ | y2y1y3 | (Lengths are not same)

Hence, it can be said that this Post Correspondence Problem is undecidable

Computational complexity

In computer science, the computational complexity or simply complexity of an algorithm is the

amount of resources required to run it. Particular focus is given to time and memory
requirements. The complexity of a problem is the complexity of the best algorithms that allow
solving the problem.

Time Complexity

Time Complexity of an algorithm is the representation of the amount of time required by the
algorithm to execute to completion. Time requirements can be denoted or defined as a
numerical function t(N), where t(N) can be measured as the number of steps, provided each
step takes constant time.

For example, in case of addition of two n-bit integers, N steps are taken. Consequently, the total
computational time is t(N) = c*n, where c is the time consumed for addition of two bits. Here, we
observe that t(N) grows linearly as input size increases.
P and NP Class

In Computer Science, many problems are solved where the objective is to maximize or minimize
some values, whereas in other problems we try to find whether there is a solution or not. Hence,
the problems can be categorized as follows −

Optimization Problem

Optimization problems are those for which the objective is to maximize or minimize some
values. For example,

● Finding the minimum number of colors needed to color a given graph.

● Finding the shortest path between two vertices in a graph.

Decision Problem

There are many problems for which the answer is a Yes or a No. These types of problems are
known as decision problems. For example,

● Whether a given graph can be colored by only 4-colors.

● Finding Hamiltonian cycle in a graph is not a decision problem, whereas checking a

graph is Hamiltonian or not is a decision problem.

What is Language?

Every decision problem can have only two answers, yes or no. Hence, a decision problem may
belong to a language if it provides an answer ‘yes’ for a specific input. A language is the totality
of inputs for which the answer is Yes. Most of the algorithms discussed in the previous chapters
are polynomial time algorithms.

For input size n, if worst-case time complexity of an algorithm is O(nk), where k is a constant, the
algorithm is a polynomial time algorithm.

Algorithms such as Matrix Chain Multiplication, Single Source Shortest Path, All Pair Shortest
Path, Minimum Spanning Tree, etc. run in polynomial time. However there are many problems,
such as traveling salesperson, optimal graph coloring, Hamiltonian cycles, finding the longest
path in a graph, and satisfying a Boolean formula, for which no polynomial time algorithms is
known. These problems belong to an interesting class of problems, called the NP-Complete
problems, whose status is unknown.
In this context, we can categorize the problems as follows −

P-Class

The class P consists of those problems that are solvable in polynomial time, i.e. these problems
can be solved in time O(nk) in worst-case, where k is constant.

These problems are called tractable, while others are called intractable or superpolynomial.

Formally, an algorithm is polynomial time algorithm, if there exists a polynomial p(n) such that
the algorithm can solve any instance of size n in a time O(p(n)).

Problem requiring Ω(n50) time to solve are essentially intractable for large n. Most known
polynomial time algorithm run in time O(nk) for fairly low value of k.

The advantages in considering the class of polynomial-time algorithms is that all reasonable
deterministic single processor model of computation can be simulated on each other with
at most a polynomial slow-d

NP-Class

The class NP consists of those problems that are verifiable in polynomial time. NP is the class of
decision problems for which it is easy to check the correctness of a claimed answer, with the aid
of a little extra information. Hence, we aren’t asking for a way to find a solution, but only to verify
that an alleged solution really is correct.

Every problem in this class can be solved in exponential time using exhaustive search.

P versus NP

Every decision problem that is solvable by a deterministic polynomial time algorithm is also
solvable by a polynomial time non-deterministic algorithm.

All problems in P can be solved with polynomial time algorithms, whereas all problems in NP - P
are intractable.

It is not known whether P = NP. However, many problems are known in NP with the property
that if they belong to P, then it can be proved that P = NP.

If P ≠ NP, there are problems in NP that are neither in P nor in NP-Complete.

The problem belongs to class P if it’s easy to find a solution for the problem. The problem
belongs to NP, if it’s easy to check a solution that may have been very tedious to find.
NP Hard and NP-Complete Classes

A problem is in the class NPC if it is in NP and is as hard as any problem in NP. A problem is
NP-hard if all problems in NP are polynomial time reducible to it, even though it may not be in
NP itself.

If a polynomial time algorithm exists for any of these problems, all problems in NP would be
polynomial time solvable. These problems are called NP-complete. The phenomenon of
NP-completeness is important for both theoretical and practical reasons.

Definition of NP-Completeness

A language B is NP-complete if it satisfies two conditions

B is in NP

Every A in NP is polynomial time reducible to B.

If a language satisfies the second property, but not necessarily the first one, the language B is
known as NP-Hard. Informally, a search problem B is NP-Hard if there exists some
NP-Complete problem A that Turing reduces to B.

The problem in NP-Hard cannot be solved in polynomial time, until P = NP. If a problem is
proved to be NPC, there is no need to waste time on trying to find an efficient algorithm for it.
Instead, we can focus on design approximation algorithm.

NP-Complete Problems

Following are some NP-Complete problems, for which no polynomial time algorithm is known.

Determining whether a graph has a Hamiltonian cycle

Determining whether a Boolean formula is satisfiable, etc..

NP-Hard Problems

The following problems are NP-Hard

The circuit-satisfiability problem

 Set Cover
Vertex Cover
Travelling Salesman Problem

Cook’s Theorem

Stephen Cook presented four theorems in his paper “The Complexity of Theorem Proving
Procedures”. These theorems are stated below. We do understand that many unknown terms
are being used in this chapter, but we don’t have any scope to discuss everything in detail.

Following are the four theorems by Stephen Cook −

Theorem-1
If a set S of strings is accepted by some non-deterministic Turing machine within polynomial
time, then S is P-reducible to {DNF tautologies}.

Theorem-2
The following sets are P-reducible to each other in pairs (and hence each has the same
polynomial degree of difficulty): {tautologies}, {DNF tautologies}, D3, {sub-graph pairs}.

First, he emphasized the significance of polynomial time reducibility. It means that if we

have a polynomial time reduction from one problem to another, this ensures that any
polynomial time algorithm from the second problem can be converted into a
corresponding polynomial time algorithm for the first problem.

Second, he focused attention on the class NP of decision problems that can be solved in
polynomial time by a non-deterministic computer. Most of the intractable problems
belong to this class, NP.
Third, he proved that one particular problem in NP has the property that every other
problem in NP can be polynomially reduced to it. If the satisfiability problem can be
solved with a polynomial time algorithm, then every problem in NP can also be solved in
polynomial time. If any problem in NP is intractable, then satisfiability problem must be
intractable. Thus, satisfiability problem is the hardest problem in NP.

Fourth, Cook suggested that other problems in NP might share with the satisfiability
problem this property of being the hardest member of NP.