SCOB000 2020

CHAPTER 4: THEORY OF COMPUTATION

It is imperative for a programmer to appreciate the crucial importance of understanding algorithms. In
other words to be able to design any program, an algorithm derived from the problem statement
should be described, which means that all executable programs have algorithms. Any problem
statement without an algorithm cannot have a computer program. That is, without an algorithm a
problem cannot be computerized.

4.1. Functions and their computations

The main goal of this chapter is to investigate the capabilities of computers. In other words, we want to
understand what machines can and cannot do and what features are required for machines to reach
their full potential. We begin with the concept of computing function.

Figure 4.1 a function that coverts measurements in yards into meters

A function in its mathematical sense is a correspondence between a collection of possible input values
and a collection of output values so that each possible input is assigned a unique output see figure 4.1.
The figure presents an example of the function that converts measurements in yards into meters. With
each measurement in yards, the function assigns the value that would result if the same distance were
measured in meters.

Another example is the addition function whose inputs are pairs of values and whose outputs are
values representing the sum of each input pair. The process of determining the particular output value
that a function assigns to a given input is called computing the function

The ability to compute functions is important because it is by means of computing functions that we are
able to solve problems. To solve an addition problem, we must compute the addition function; to sort a
list, we must compute the sort function. In turn, a fundamental task of computer science is to find
techniques for computing the functions that lie beneath the problems we want to solve.

Non-computable function – is a function that cannot be computed by any algorithm. In other words,
there are functions that are so complex that there is no well-defined, step-by-step process for
determining their outputs based on their input values.

A function is computable if there is a way of calculating output values. To calculate means a procedural,
step-by-step process or, an algorithm do exist.

- 77 -
SCOB000 2020

Since machines can only perform tasks described by algorithms, the study of computable functions is
the study of the ultimate capabilities of machines.

4.2. Turing Machines

We must recall that to have a machine perform a task, we must first find an algorithm for performing
that task. Thus, since there are no algorithms for computing non-computable functions, these functions
lie beyond the powers of today’s as well as tomorrow’s computers. In an effort to understand such
limitations of machines, Alan Turing designed a machine called, Turing Machine, which we can still use
even today as a tool for studying the power of algorithmic process.

 Turing Machine fundamentals

A Turing machine consists of a control unit that can read and write symbols on a tape by means of a
read/write head, figure 4.2. The tape extends indefinitely at both ends and is divided into cells, each of
which can contain any one of a finite set of symbols. The set is called the machine’s alphabet. At any
time during machine’s computation, the machine must be in one of a finite number of conditions,
called states. A Turing machine’s computation begins in a special state called the start states. A Turing
machine’s computation consists of a sequence of steps that are executed by the machine’s control unit.
Each step consists of observing the symbol in the current tape cell (the one viewed by the read/writ
head), writing a symbol in that cell, possibly moving the read/write head one cell to the left or right,
and then changing states. The exact action to be performed is determined by a program that tells the
control unit what to do, based on the machine’s state and the contents of the current tape cell.

Figure 4.2 components of a Turing machine

Let us now consider an example of a specific Turing machine. For this purpose, we represent the
machine’s tape as a horizontal strip divided into cells in which we can record symbols from the
machine’s alphabet. We indicate the current position of the machine’s read/write head by placing a
pointer under the appropriate tape cell. The alphabet for our example consists of the symbols 0, 1 and
*. The tape of our machine appears as follows:

* 1 0 1 *

Current position

By interpreting a string of symbols on the tape as representing binary numbers separated by asterisks,
we recognize that this particular tape contains the value 5. Our Turing machine is designed to
increment such a value on the tape by 1. More precisely, it assumes that the starting position is at an
asterisk marking the right end of a string of 0s and 1s, and it proceeds to alter the bit pattern to the left

- 78 -
SCOB000 2020

so that it represents the next larger inter. The sates for our machine are START, ADD, CARRY,
OVERFLOW, RETURN, and HALT.

Figure 4.3, a Turing machine for implementing a value

The actions corresponding to each of these states and the content of the current cell are described in
the table of figure 4.3. We assume that the machine always begins in the START state. Let us apply this
machine to the tape pictured earlier that contains the value 5. Observe that when in the start state with
the current cell containing * (as in our case), we are instructed by the table to rewrite the *, move the
read/write head one cell to the left, and enter the ADD state. Having done this, the machine can be
described as follows:

* 1 0 1 *

Machine sate = ADD Current position

To proceed, we look at the table to see what to do when in the ADD state with the current cell
containing 1. The table tells us to replace the 1 in the current cell with 0, move the read/write head one
cell to the left, and enter the CARRY state. Thus the configuration of the machine becomes:

* 1 0 1 *

Machine state = CARRY Current position

We again refer to the table to see what to do next and find that when in the CARRY state with the
current cell containing 0, we should replace the 0 with 1, move the read/write head one cell to the
right, and enter the RETURN state. After doing this the machine’s configuration is as follows

- 79 -
SCOB000 2020

* 1 1 0 *

Machine state = RETURN Current position

From this situation, the table instructs us to proceed by replacing the 0 in the current cell with another
0, move the read/write head one cell to the right, and remain in the RETURN state. Consequently, we
find our machine in the following condition:

* 1 1 0 *

Machine state = RETURN Current position

At this point, we see that the table instructs us to rewrite the asterisk in the current cell and HALT. The
machine thus stops in the following configuration (the symbols on the tape now represent the value 6
as desired)

* 1 1 0 *

Machine state = HALT Current position

4.2.1 The Church-Turing Thesis

The Turing machine in the preceding example can be used to compute the function known as the
successor function, which assigns each nonnegative integer input value n to the output value n + 1. We
need merely place the input value in its binary form on the machine’s tape, run the machine until it
halts, and then read the output value from the tape.

A function that can be computed in this manner by a Turing machine is said to be Turing computable.

Turing’s conjecture was that the Turing-computable functions were the same as the computable
functions. In other words, he conjectured that the computational power of the Turing machines
encompasses that of any algorithmic system or, equivalently, that the Turing machine concept provides
a context in which solutions to all the computable functions can be expressed. Today, this conjecture is

- 80 -
SCOB000 2020

often referred to as the Church-Turing thesis, in reference to the contributions made by both Alan
Turing and Alonzo Church.
If a computational system is capable of computing all the Turing computable functions, it is considered
to be as powerful as any computational system can be.

Exercises
1. Apply the Turing machine described in this section, starting with the following initial status:

* 1 1 0 *

Machine state = START Current position

2. Describe a Turing machine that replaces a string of 0s and 1s with a single 0

3. Describe a Turing machine that decrements the value on the tape if it is greater than zero or
leaves the value unaltered if it is zero.

4.3. Universal Programming Language

In this section we describe a simple imperative programming language that is rich enough to allow us to
express programs for computing all computable functions. However, if you find that a particular
problem cannot be solved using this language, the reason will not be a fault of this language, but rather
that there is no an algorithm for solving such a problem. It is important to note that a universal
language needs not to be complex. Indeed, the language we present in this section is very simple.

4.3.1 The Bare Bones Language

We begin the presentation of Bare Bones by considering the declarative statements found in other
languages. These statements allow programmers the luxury of thinking in terms of data structures and
data types (such as arrays and strings) even though the machine itself just manipulates bit patterns
without knowledge of what patterns represent. Before being presented to a machine for execution, a
high-level instruction dealing with elaborate data types and structures must be translated into
machine-level instructions that manipulate bit patterns to simulate the action requested. Thus the
design of a programming language can be simplified by forcing the programmer to express all
operations in terms of bit patterns in the first place. Such a language would have a single data type and
data structure, so it would not need data description statements.

Now to make things simpler, for our Bare Bones language we will adopt this approach. All variables are
considered to be of type “bit pattern of arbitrary length.” Thus in a Bare Bone program we do not need
declarative statements by which variable names and their associated properties are described; a
programmer can simply begin using a new variable name when it is needed, with the understanding
that it refers to a bit pattern of arbitrary length. The translator for the Bare Bone language however,
must be able to distinguish variable names from other terms. This is done by designing the syntax of
Bare Bones so that, the role of any term can be identified by syntax alone. For this purpose, we specify

- 81 -
SCOB000 2020

that variable names must begin with a letter from the English alphabets, which can be followed by any
combination of letters and digits.

As for imperative statements, Bare Bones contain three assignments statements and one loop
structure. It is a free format language, so each of these statements terminates with a semi-colon,
making it easy for a translator to separate statements that may appear on the same line. We, however,
will adopt the policy of writing only one statement per line to enhance readability. Each of the three
assignment statements request that the contents of the variable identified in the statement be
modified. The first allows us to associate a string of zeros with a variable.
Its syntax is
clear name;
where name can be any variable name.
The other assignment statements are essentially opposites of each other:
incr name;
and
decr name;
Again, name represents any variable name. The first of these statements causes the value associated
with the identified variable to be incremented. Here the term increment refers to the interpretation of
bit patterns as representing numeric values in base two notation and means to change the pattern to
represent the next large inter.

To illustrate, if the pattern 101 is associated with the variable Y before the statement

incr Y;
is executed, then the pattern 110 is associated with Y afterwards. That is, 1 is added to the value
assigned to Y. In contrast, the decr statement is used to decrement the value associated with the
identified variable or, in other words, to decrease the represented value by one. An exception is when
the variable is already associated with zero, in which case this statement leaves the value unaltered.
Therefore, if the value associated with Y is 101 before the statement
decr Y;
is executed, the pattern 100 is associated with Y afterward. However, if the value of Y had been zero
before executing the statement, the value would remain zero after execution.

The Bare Bones programming language contains only one control structure represented by a while-end
statement pair. The statement sequence
while name not 0 do;
.
.
.
end;
(where name represent any variable name) causes any statement or statement sequence positioned
between the while and end statement to be repeated as long as the value of the variable name is not
zero. To be more precise, when a while-end structure is encountered during program execution, the
value of the identified variable is first compared to zero. If it is zero, the structure is skipped and
execution continues with the statement sequence following the end statement. If, however, the
variable’s value is not zero, the statement sequence within the while-end structure is executed and
control is returned to the while statements, whereupon the comparison is conducted again. Note that
the burden of the loop control is partially placed on the programmer, who must explicitly request that
the variable’s value be altered within the loop body to avoid an infinite loop. For instance, the sequence
incr X;
while X not 0 do;

- 82 -
SCOB000 2020

incr Z;
end;
results in an infinite process because once the while statement is reached, the value associated with X
can never be zero, whereas the sequence
clear Z;
while X not 0 do;
incr Z;
decr X;
end;
ultimately terminates with the effect of transferring the value initially associated with X to the variable
Z.

Observe that the while and the end statements must appear in pairs with the while statements
appearing first. However, a while-end statements pair may appear within the instructions being
repeated by another while-end pair. In such a case the pairing of while and end statements is
accompanied by scanning the program in its written form from beginning to end while associating each
end statement with the nearest while statements not yet paired. Although not syntactically necessary,
we often use indentation to enhance the readability of such structures.

As an example, the instruction sequence in figure 4.4 results in the product of the values associated
with X and Y being assigned to Z, although it has the side effect of destroying any nonzero value that
may have been associated with X. (The while-end structure controlled by the variable W has the effect
of restoring the original value of Y.)

Figure 4.4 A Bare Bone program for computing X * Y

4.3.2 Programming in Bare Bones

The main goal of presenting Bare Bones language is to investigate what is possible, not what is practical.
Now, let us just demonstrate how Bare Bone can be used to express some elementary operations.

We first note that with a combination of the assignment statements, any value (bit pattern) can be
associated with a given variable. For example, the following sequence assigns the pattern 11 (binary
representation for 3) to the variable X by first clearing any previous association and then incrementing
its value three times:
clear X;
incr X;
incr X;
incr X;

- 83 -
SCOB000 2020

Another common activity in programs is to copy data from one location to another. In terms of Bare
Bones, this means that we need to be able to assign to one variable a bit pattern previously assigned to
another. This can be accomplished by first clearing the destination and then incrementing it an
appropriate number of times. In fact, we have already observed that the sequence
clear X;
while X not 0 do;
incr Z;
decr X;
end;
transfers the value associated with X to Z. however, this sequence has the side effect of destroying the
original value of X. To correct this, we can introduce an auxiliary variable to which we first transfer the
subject value from its initial location. We then use this auxiliary variable as the data source from which
we restore the original variable while placing the subject value in the desired destination. In this
manner, the movement of today to yesterday can be accomplished by the sequence as shown in figure
4.5.

Figure 4.5 A Bare Bone implementation of the instruction “copy Today to Tomorrow”

We adopt the syntax

Copy name1 to name2;
(where name1 and name2 represent variable names) as a short-hand notation for a statement
structure of the form in figure 4.5. Thus, although Bare Bones itself does not have an explicit copy
instruction, we often write programs as though it did, with the understanding that to convert such
informal programs into real Bare bones programs, we must replace the copy statements with their
equivalent while-end structure using an auxiliary variable whose name does not clash with a name
already used elsewhere in the program.

4.3.3 The Universality of Bare Bones

We now apply the Church-Turing thesis to confirm the claim that Bare Bones is a universal
programming language. First, we observe that any program written in Bare Bones can be thought of as
directing the computation of a function. The function’s input consists of the values assigned to variables
prior to executing the program, and the function’s output consists of the values of variables when the
program terminates. To compute the function, we merely execute the program, starting with proper
variable assignments, then observe the variables’ values when the program terminates.

Under these conditions, the program

incr x;

- 84 -
SCOB000 2020

directs the computation of the same function (the successor function) that is computed by the Turing
machine example of section 3.2. Indeed, it increases the value associated with X by 1. Likewise, if we
interpret the variables X and Y as inputs and the variable Z as the output, the program
copy Y to Z;
while X not 0 do;
incr Z;
decr X;
end;
directs the computation of the addition function

So far research shows that Bare Bones programming can be used to express algorithms for computing
all the Turing-computable functions. Combining this with the Church-Turing thesis implies that any
computable function can be computed by a program written in Bare Bones. Thus, Bare Bones is a
universal programming language in the sense that if an algorithm exists for solving a problem, then that
problem can be solved by some Bare Bones programs. In turn, Bare Bones could theoretically serve as a
general-purpose programming language.

4.4 A Non-Computable Function

We now identify a function that is not Turing computable and so, by the Church-Turing thesis, is widely
believed to be non-computable in the general sense. In other words, it is a function whose computation
lies beyond the capabilities of today’s computer.

4.4.1 The Halting problem

The non-computable function we are now introducing is associated with a problem known as the
halting problem, which (in an informal sense) is the problem of trying to predict in advance whether a
program will terminate (or halt) if started under certain conditions. For example, consider the simple
Bare Bone program
while X not 0 do;
incr X;
end;
if we execute this program with the initial value of X being 0, the loop will not be executed and the
program’s execution will quickly terminate. However, if we execute the program with any other initial
value of X, the loop will be executed forever, leading to a non-termination process. In this case, then, it
is easy to conclude that the program’s execution will halt only when it is started with X assigned the
value 0. However, as we move to more complex examples, the task of predicting a program’s behavior
becomes more complicated. In fact, in some cases the task is impossible, as well shall see. But first we
need to formalize our terminology and focus our thoughts more precisely.

Our example has shown that whether a program ultimately halts can depend on the initial values of its
variables. Thus if we hope to predict whether a program’s execution will halt, we must be precise in
regard to these initial values. The choice we are about to make for these values may see strange to you
at first, but do not despair. Our goal is to take advantage of a technique called self-reference – the idea
of an object referring to itself.

In our case, self-reference will be achieved buy assigning a program’s variables an initial value
representing the program itself. To this end observe that each Bare Bones program can be encoded as a
single, long, bit pattern in a one-character-per-byte format using ASCII. Moreover, each variable in a
Bare Bones program is of type “bit pattern of arbitrary length,” so we can assign this encoded version of
a program as the value of its variables.

- 85 -
SCOB000 2020

Let us consider what would happen if we did this in the case of simple program
while X not 0 do;
incr X;
end;
We want to know what would happen if we started this program with X assigned the encoded version
of the program itself figure 4.6. in this case the answer is readily apparent. Since X would have a
nonzero value, the program would become caught in the loop and never terminate. One the other
hand, if we performed a similar experiment with the program
clear X
while X not 0 do;
incr X;
end;
the program would terminate since the variable X will have the value 0 by the time the while-end
structure is reached regardless of its initial value.

Let us then, make the following definition: A bare Bones program is self terminating if executing the
program with all of its variables initialized to the program’s own encoded representation leads to a
termination process. Informally, a program is self terminating if its execution terminates when started
with itself as its input. Here, then, is the self-reference that we promised

Figure 4.6 testing a program for self-termination

Let’s note that whether a program is self-terminating probably has nothing to do with the purpose for
which the program was written. It is merely a property that each Bare Bones program either possesses
or does not possess. That is, each Bare Bones program is either self-terminating or not. We now can
define a halting problem in a precise manner. Halting problem - is the problem of determining whether
Bare Bones programs are or are not self-terminating.

Note that, there is no single algorithm that, when given any Bare Bones program, is capable of
determining whether that program is or is not self-terminating. Thus the solution to the halting
problem lies beyond the capabilities of computers.

Exercise

1. Assuming that the function’s input is represented by X and its output by Z, write a Bare Bones
program that takes a number then multiply it by 2.
2. Show that the statement invert X; (whose action is to convert the value of X to zero if its initial
value is nonzero and to 1 if its initial value is zero) can be simulated by a Bare Bones program
segment.

- 86 -
SCOB000 2020

4.5 The Un-solvability of the Halting Problem

Let us now show that solving the halting problem lies beyond the capabilities of machines. Our
approach is to show that to solve the problem would require an algorithm for computing non-
computable function. The inputs of the function in question are encoded versions of Bare Bones
programs; its outputs are limited to the values 0 and 1. More precisely, we define the function so that
the representation of a self-terminating program produces the output value 1 and the representation of
a non-self-terminating program produces the output value 0. For the sake of conciseness, we will refer
to this function as the halting function. Our main task is to show that the halting function is not
computable. The approach uses the technique known as “proof by contradiction.” In short, we prove
that a statement is false by showing that it cannot be true. Let us, then, show that the statement “the
halting function is computable” cannot be true. The entire argument is summarized in figure 4.7.

Figure 4.7, proving the insolvability of the program

If the halting function is computable, then (since Bare Bones is a universal programming language)
there must be a Bare Bones program that computes it. In other words, there is a Bare Bones program

- 87 -
SCOB000 2020

that terminates with its output equal to 1 if its input is the encoded version of self-terminating program
and terminates with its output equal to 0 otherwise. To apply this program we need not identify which
variable is the input variable but instead merely initialize all the program’s variables to the encoded
representation of the program to be tested. This is because a variable that is not an input variable is
inherently a variable whose initial value does not affect the ultimate output value. We conclude that if
the halting function is computable, then there is a Bare Bones program that terminates with its output
equal to 1 if all its variables are initialized to the encoded version of a self-terminating program and
terminates with its output equal to 0 otherwise.

Assuming that the program’s output variable named X (if it is not we could simply rename the
variables), we could modify the program by attaching the statements
while X not 0 do;
end;
at its end, producing a new program. This new program must be either self-terminating or not.
However, we are about to see that it can be neither. In particular, if this new program was self-
terminating and we ran it with its variables initialized to the program’s own encoded representation,
then when its execution is reached the while statement that we added, the variable X would contain a
1. (To this point the new program is identical to the original program that produced a 1 if its input was
the representation of a self-terminating program.) At this point, the program’s execution would be
caught forever in the while-end structure because we made no provision for X to be decremented
within the loop. But this contradicts our assumption that the new program is self-terminating.
Therefore we conclude that new program is not self-terminating.
If, however, the new program was not self-terminating and we executed it with its variables initialized
to the program’s own encoded representation, it would reach the added while statement with X being
assigned to the value 0. (This occurs because the statements preceding the while statement constitute
the original program that produces an output of 0 when its input represents a program that is not self-
terminating.) In this case, the loop in the while-end structure would be avoided and the program would
halt. But this is the property of a self-terminating program, so we are forced to conclude that the new
program is self-terminating, just as we were forced to conclude earlier that it was not self-terminating.
In short, we see that we have the impossible situation of a program that on the one hand must be
either self-terminating or not and on the other hand can be neither. Consequently, the assumption that
led to this dilemma must be false.

We conclude that the halting function is not computable, and since the solution to the halting problem
relies on the computation of that function we must conclude that solving the halting problem lies
beyond the capabilities of any algorithmic system. Such problems are called unsolvable problems.

- 88 -
SCOB000 2020

Exercise
1. Is the following Bare Bones program self-terminating? Explain
copy X to Y;
incr Y;
incr Y;
while X not 0 do;
decr X;
decr X;
decr Y;
decr Y;
end;
decr Y;
while Y not 0 do;
end;

(To substantiate your answer, give X some value then draw a trace table indicating the state of X and Y
in each and every execution.)

4.6 Public Key Cryptography

In this section we investigate encryption techniques. It involves values known as keys that are used to
encrypt data and to decode encrypted data.

A Key is a value used to encrypt or decrypt a message

– Public key: Used to encrypt messages
– Private key: Used to decrypt messages
RSA algorithm is a popular public key cryptographic algorithm
– Relies on the (presumed) intractability of the problem of factoring large numbers
Such encryption techniques compose a field of study known as public key cryptography.

4.6.1 Encryption via Knapsack problem

To describe a specific public key encryption system, we begin with the non-deterministic polynomial
(NP) complete problem known as the knapsack problem. This is the problem of selecting numbers from

- 89 -
SCOB000 2020

a collection so that the sum of the selected numbers is a particular value. It is called the knapsack
problem because it is analogous to the problem of selecting a collection of its items that exactly fill a
knapsack. An example of a knapsack problem is the problem of selecting the collection of values from
the list

191 691 573 337 730 651 493 177 354

whose sum is 2063

The best known method for solving knapsack problems in general is to try all possible combinations
systematically until a solution is found. But, if there are n values from which to select, there are 2 n
different combinations to test. Thus, unless we are lucky and find the correct combination early in our
search, the time required to find the solution will be quite large. To appreciate this phenomenon for
yourself, try solving the knapsack problem previously mentioned. You will find that it can be rather
time-consuming even though there are only 10 values from which to choose. Imagine your frustration if
there were 20 values from which to select and therefore more than a million combination to test.

We can use knapsack problem based on the list above to encrypt messages as follows: we first
represent a message as a string of bits, perhaps using ACSII or Unicode. Then, we break this string into
segments of 10 bits each and represent each 10-bit segment by a single number. This number is
obtained by adding the values from the list that occupy the positions corresponding to the positions of
the 1s in the 10-bit segment.
For instance, the 10-bit segment:

1001100001
would be represented by 1247, figure 4.8. This is because the 1s in the segment are found in the first,
fourth, fifth, and the tenth positions, and the sum of the corresponding values in the list (191, 337, 365,
and 354) is 1247.
Likewise the pattern 0010011010 would be represented by 2131 (which is 573 + 730 + 651 + 177).

Figure 4.8 encrypting a bit pattern as a knapsack problem

In turn, the message

10011000010010011010

would be encrypted as the two-number sequence 1247 followed by 2131.

Suppose someone intercepted this encrypted message and suppose that person even knew the list of
values that was used to encrypt it. That person would still have to solve two knapsack problems to
decode the message, which could be a time-consuming process. Moreover, if the size of the list used to
encrypt messages was significantly larger than 10, the task of decoding intercepted messages would be
completely intractable – which means the contents of the message would be secure.

- 90 -
SCOB000 2020

The problem with this simple system is that no one would be able to decode the message quickly – not
even the addressee. What we need is a trick that would allow the addressee to solve the knapsack
problems quickly, while everyone else is faced with an unbearably time-consuming task.
To obtain such a trick, we observe that some knapsack problems are easy to solve. Suppose the values
from which we are asked to select are:

1 4 6 12 24 51 105 210 421 850

each number in this list is larger than the sum of the preceding numbers. So, if we needed to select a
collection of values whose sum is 995, we would know immediately that 850 must be one of the
required values because the sum of all the other values would be too small. Having made this selection,
our problem would be reduced to selecting numbers whose sum is 995 – 850, or 145. But this means
that we must select 105 because the sum of the other possible values would be smaller than 145.
Continuing in this fashion, we could quickly conclude that the values to be selected are 850, 105, 24, 12,
and 4.

And now for the punch line - there is a way of converting such easy knapsack problems into hard
knapsack problems and back again. For now, we will consider this conversion process in terms of three
“magic” numbers. Later we will consider the origin of these magic numbers and how you can build your
own encryption system. The magic numbers we will use are:
642, 2311, and 18
Our fist step is to convert the list:

1 4 6 12 25 51 105 210 421 850

with which easy knapsack problems are constructed into another list in terms of which knapsack
problems are more difficult. This we do by multiplying each entry in the list by 624 (the first magic
number), dividing these products by 2311 (the second magic number), and recording the remainders
from these division problems. This produces the list:
642 257 1541 771 2184 388 391 782 2206 304

In particular the value 4 in the original list is replaced by 257 in the new list, because 4*642 = 2568 and
2568/2311, produces a remainder of 257.

Observe that a knapsack problem posed in terms of this new list would be difficult since our translation
process has destroyed the relationship that existed among the values in the original list. But by knowing
the magic numbers, we can solve such problems quickly. Our approach is to multiply the target sum by
18 (the third magic number), divide the product by 2311 (the second magic number), and record the
remainder from this division. We then use this remainder as the target sum in a knapsack posed in
terms of the original easy knapsack system. Once this easy problem is solved, the values in the original
list that solve the original knapsack problem are those in the positions that correspond to the solution
to the easy knapsack problem.
For example,
Suppose the problem was to select the values from the list

624 257 1541 771 2184 388 391 782 2206 304
whose sum is 4895.
We first compute 4895 *18 = 88110, then divide 88110 by 2311 to obtain the remainder 292. Next we
determine that the values 6, 25, 51, and 210 are the values in the list

1 4 6 12 25 51 105 210 421 850

whose sum is 292.

- 91 -
SCOB000 2020

Since these are the third, fifth, sixth, and eighth values in their respective list, we conclude that the
third, fifth, sixth, and eighth entries in the list

624 257 1541 771 2184 388 391 782 2206 304

are the ones whose sum is 4895. Indeed,

1541+2184+388+782 = 4895,

as desired. The entire public key encryption system works as follows: we openly distribute the list

624 257 1541 771 2184 388 391 782 2206 304

and allow people to encrypt messages in terms of knapsack problems based on this list. But, we keep
the original list as well as the three secret numbers to ourselves. As we receive encrypted messages, we
decode them quickly by converting them into easy knapsack problems, but no one else can do so. Thus,
the messages sent to us are secure figure 4.9.

Figure 4.9 public key encryption using knapsack problem

4.6.2 Modular Arithmetic

The public key encryption system just described relies on a mathematical concept known as modular
arithmetic. A modular arithmetic system is merely a system obtained by substituting each integer in the
traditional system with the remainder that is obtained by dividing the integer by a predetermined
value. This predetermined value is called the modulus. For example, if we pick 7 to be the modulus,
then the integer values,
0 1 2 3 4 5 6 7 8 9….
Would be translated into the values
0 1 2 3 4 5 6 0 1 2….

It is customary to use the notation x (mod m), which is read “x mod m” to represent the remainder
obtained when the value x is divided by m. Thus 9 mod 7 is 2, 24 mod 7 is 3 and 5 mod 7 is 5.

Two integers that produce the same remainder when divided by m are said to be equivalent modulo m.
Thus 16 and 23 are equivalent modulo 7 because 16 mod 7 is the same as 23 mod 7. Indeed, both 16

- 92 -
SCOB000 2020

and 23 produce the remainder 2 when divided by 7. it is customary to use the notation x ≡ y (mod m) –
read “x is equivalent to y, mod m” – to mean that x is equivalent to y when using the modulus m. Thus
16 ≡ 23 (mod 7)

After translating the traditional integer values into a modular system using m as the modulus, we are
left with only the values 0, 1, 2, 3, …, m-1. We can perform arithmetic within this restricted set of values
by first performing an operation as in traditional arithmetic and then translating the answer, say x, back
into the restricted range by replacing it with the value x mod m. Thus in a modular system based on the
modulus 7, we would be restricted to the values 0, 1, 2, 3, 4, 5 and 6. The sum 2 + 6 would be 1 since
2+6 = 8, which produces the remainder 1 when divided by 7. Moreover 6*2 would 5.

Arithmetic within a modular system is therefore a distorted reflection of arithmetic in the tradition
system. It is a reflection in the sense that if x ≡ a (mod m) and y ≡ b (mod m), then x + y ≡ a + b (mod m).
But it is also a distortion in the sense that sums and products are not the same in the two systems. In
particular, the product of two distinct integers can be 1 in a modular system, a phenomenon that does
not occur in the tradition system on integers. For example, in the modular system based on the
modulus 7, we have 3*5 = 1. Two numbers that produce the product 1 are called multiplicative inverse
of each other. Within the traditional system of integers the value 3 does not have multiplicative inverse.
Instead, the traditional multiplicative inverse of 3, which is 1/3, lies outside the system of integer
values. But in the system of integers modulo 7, the value 3 does have a multiplicative inverse which is 5.

Mathematics tells us that if x and m are two positive integers such that x<m and x and m are relatively
prime (meaning that the only integer that evenly divided both x and m is 1), then the value x will have a
multiplicative inverse in the modular system based on the modulus m. For example, 6 is less than 13
and the two values have no divisors other than 1 in common. Thus 6 must have a multiplicative inverse
in the system with modulus 13. In fact, its inverse is 11 since 6*11 = 66, which when divided by 13
leaves a remainder of 1. That is, 6*11 ≡ 1 (mod 13).

 Encryption

Note that if the value x is nonnegative and less than the modulus m, then x (mod m) is x itself. This
means that as long as we perform arithmetic operations whose traditional results fall within the range
of 0 to m-1, the results obtained will agree with those in the modular system. Thus, if we pick an
extremely large modulas, we could perform our daily arithmetic computations without ever knowing
whether we were in the traditional arithmetic system or a modular system. In particular, since the sum
of all the values in the list

1 4 6 12 25 51 105 210 421 850

is 1685,

The additions performed when trying to solve a knapsack problem based on this list will never produce
results larger than 1685. When solving these problems, we thus, do not need to be concerned with
whether we are working within the traditional arithmetic system or a modular system whose modulus
is greater than 1685. On the other hand, if we pretend that our easy knapsack problem is posed in such
a large modular system, we can contain a method of converting it into a more difficult problem and
back again. To explain, let us suppose that we have a list of values

a1 a2 a3 a4 a4 a5 a6 a7 a8 a9 a10
such that each entry in the list is larger than the sum of its predecessor. It is, therefore, a list in terms of
which knapsack problems are easily solved. Let us pick a modulus m that is larger than the sum of all

- 93 -
SCOB000 2020

the values in this list and pick two other values x and y that are multiplicative inverses in the modular
system based on the modulus m.
If we multiply each entry in our original list by x, we obtain the list

a1x a2x a3x a4x a4x a5x a6x a7x a8x a9x a10x
in terms of which knapsack problems are again easily solved. (each entry is till larger than the sum of
those preceding it). But let us now exchange each entry in our new list with a value that is equivalent to
it modulo m. In particular, in place of a1x we will put the value a1x (mod m), in place of a2x we will put
the value a2x (mod m), and so on. This will produce the list

b1 b2 b3 b4 b5 b6 b7 b8 b9 b10
where each entry is equivalent modulo m to the corresponding entry in the list

a1x a2x a3x a4x a4x a5x a6x a7x a8x a9x a10x
In turn, any sum of values from this new list must be equivalent modulo m to the sum of the
corresponding values in the list
a1x a2x a3x a4x a4x a5x a6x a7x a8x a9x a10x
Suppose, then that we are given a sum such as
b1 + b2 + b5
and asked to select the entries from the list

b1 b2 b3 b4 b5 b6 b7 b8 b9 b10
that produce that sum.
Since:
b1 + b2 + b5 ≡ (a1x + a2x + a5x) (mod m)
and y is the multiplicative inverse of x, we know that

y (b1 + b2 + b5 ) ≡ y (a1x + a2x + a5x) (mod m)

≡ yx (a1 + a2 + a5) (mod m)
≡ (a1 + a2 + a5) (mod m)
This means that if we multiply a sum of values selected from the list

b1 b2 b3 b4 b5 b6 b7 b8 b9 b10
by y, divide the product by m, and record the remainder, then that remainder will be the sum of the
corresponding entries in the original list

a1 a2 a3 a4 a4 a5 a6 a7 a8 a9 a10
But, since knapsack problems are easily solved in terms of this list, we can quickly discover what these
entries are. In turn, we can solve the original knapsack problem by selecting the corresponding entries
from the list

b1 b2 b3 b4 b5 b6 b7 b8 b9 b10

In short, to select the values from the above list that were used to form the sum s, we need merely
compute the values s*y (mod m), then find the entries in the list

a1 a2 a3 a4 a4 a5 a6 a7 a8 a9 a10
whose sum is this value, and then select the corresponding values from the list

b1 b2 b3 b4 b5 b6 b7 b8 b9 b10

- 94 -
SCOB000 2020

As an example, let us start with the list

1 4 6 12 25 51 105 210 421 850

In terms of which knapsack problem are easily solved. Since the sum of all the values in this list is 1685,
the value 2311 is sufficiently large to play the role of m. Moreover, 642 and 18 are multiplicative
inverses in the modular system using modulus 2311, so let us use 642 for the value x and 18 for the
value y. Our first step is to multiply each entry in the list above by 642 and record the remainder
obtained by dividing this product by 2311. This produces the list

642 257 1541 771 2184 388 391 782 2206 304

Suppose we are now given the problem of selecting the values in this list whose sum is 4507. we
multiply 4507 by 18 to obtain 81126, divide this value by 2311, and record the remainder, which is 241.
Then, we find that the values 6, 25, and 210 are the numbers in the original list whose sum is 241. since
these are the third, fifth, and eighth entries in their respective list, we conclude that the third, fifth and
eighth entries in this list

642 257 1541 771 2184 388 391 782 2206 304
are those whose sum is 4507.
Indeed, the values 1541, 2184, and 782 solve the original knapsack problem.

In summary we can build a public key encryption system shown in figure 4.10. We first write down a list
of values from which easy knapsack problems are constructed. Next, we pick values m, x, and y such
that m is larger than the sum of all the values in the list and x is the multiplicative inverse of y in the
modular system modulo m. We then multiply the values in the original list by x, divide these products
by m, and record the remainders. The list of these remainders is the public encryption key. Anyone can
encrypt a message as a sequence of knapsack problem based on this list, and we will be the only ones
who can decode such messages easily. We need merely multiply each sum we are given by y, divide the
product by m, and record the remainder. Then, we can quickly find the values from the original list
whose sum is that remainder and reconstruct the bit pattern that formed the message.

- 95 -
SCOB000 2020

Figure 4.10 constructing a public key encryption system

Lastly, an adversary could try to break our encryption system by guessing the values m, x, and y rather
than solving the difficult knapsack problems. This is why the numbers used in an actual encryption
system should be much larger than the values we have used in our examples. By selecting large
knapsack problems and large values for the keys, the time required to solve the knapsack problem
directly or to guess the private keys can be made significant.

Exercises:
1. Find the values in the list below that produce the sum 2200

191 691 573 337 365 730 651 493 177 354
2. Find the values in the list below that produce the sum 3223

642 257 1541 771 2184 388 391 782 2206 304
3. Find the multiplicative inverse of 5 in the modular system whose modulus is 23
4. Design a public key encryption system, based on the list

2 3 6 12 24,

and the fact that 30 and 38 are multiplicative inverses in a modular system with modulus 67.

- 96 -