Theory of Computation

Lecture#24

Universal Turing Machine

blog.bazaarvoice.com
Umar Faiz
Pakistan Institute of Engineering and Applied Sciences
 TURING MACHINE
Input Tape
alphabet alphabet
States
is a finite set of symbols
containing  as its
M  (Q, , ,  , q0 , , F ) subset and it is the set of
tape symbols

Transition Final
function Initial states
blank
state
Slides based on Costas Busch - RPI
UNIVERSAL TURING MACHINES
• The Turing model is a better model for a general-purpose computer. This
model adds an extra element to the specific computing machine: the
program. A program is a set of instructions that tells the computer what to
do with data.

A computer based on the Turing model

UNIVERSAL TURING MACHINES

Same program, different data

UNIVERSAL TURING MACHINES

Same data, different programs

UNIVERSAL TURING MACHINES
• A universal Turing machine, a machine that can do any computation if the
appropriate program is provided.
• A very powerful computer and a universal Turing machine can compute
the same thing.
• We need only provide the data and the program—the description of
how to do the computation.
• A universal Turing machine is capable of computing anything that is
computable.
 UNIVERSAL TURING MACHINES
• Limitation of Standard Turing Machines:
– Standard Turing Machines execute only one program

• Real Computers are re-programmable

• Universal Turing Machines:

– Reprogrammable machine
– Simulates any other Standard Turing Machine
– Input of Universal Turing Machine
• Description of transitions of M
• Initial tape contents of
M
Slides based on Costas Busch - RPI
 UNIVERSAL TURING MACHINES
Three tapes Tape 1
Description of M
Universal
Turing
Tape 2
Machine
Tape Contents of M

Tape 3
M
State of M
Slides based on Costas Busch - RPI
 UNIVERSAL TURING MACHINES
Tape 1

Description of M

We describe Turing machine M as a string of symbols:

We encode M as a string of symbols

Slides based on Costas Busch - RPI

 UNIVERSAL TURING MACHINES
Alphabet Encoding:
Symbols: a b c d 

Encoding: 1 11 111 1111

Slides based on Costas Busch - RPI

 UNIVERSAL TURING MACHINES
State Encoding:

States: q1 q2 q3 q4 

Encoding: 1 11 111 1111

Slides based on Costas Busch - RPI

 UNIVERSAL TURING MACHINES
Head Move Encoding:

Move: L R

Encoding: 1 11

Slides based on Costas Busch - RPI

 UNIVERSAL TURING MACHINES
Transition Encoding:

Transition:  (q1, a )  (q2 , b, L)

Encoding: 1 0 1 0 11 0 11 0 1

separator

Slides based on Costas Busch - RPI

 UNIVERSAL TURING MACHINES
Machine Encoding:

Transitions:

 (q1, a )  (q2 , b, L)  (q2 , b)  (q3 , c, R )

Encoding:
1 0 1 0 11 0 11 0 1 00 11 0 1 10 111 0 111 0 11
separator
Slides based on Costas Busch - RPI
 UNIVERSAL TURING MACHINES
Tape 1 contents of Universal Turing Machine:
Encoding of the simulated machine M as a binary string of 0’s and 1’s

Slides based on Costas Busch - RPI

 UNIVERSAL TURING MACHINES
• A Turing Machine is described with a binary string of 0’s and 1’s.
• Therefore, the set of Turing machines forms a language and each string is
the binary encoding of a Turing Machine

Slides based on Costas Busch - RPI

 UNIVERSAL TURING MACHINES
Language of Turing Machines:

L = { 010100101, (Turing Machine 1)

00100100101111, (Turing Machine 2)

111010011110010101,

…… } (Turing Machine …)

Slides based on Costas Busch - RPI

Countable Sets
COUNTABLE SETS
 COUNTABLE SETS
• Natural Numbers = {1,2,3,4,5, …}
• Integers = {…, -5, -4, -3,- 2, -1, 0, 1, 2, 3, 4, 5, …}
• Rational Numbers of fractions: All the decimals which terminate or repeat.
• Irrational Numbers: All the decimals which do not terminate or repeat.
• Real Numbers: All decimals

Slides based on Costas Busch - RPI

 COUNTABLE SETS
• Infinite sets are either Countable or Uncountable
• Countable Sets:
– A set is countable if
• It is finite or it has the same cardinality as the natural numbers N.
• In any finite set or any Countably infinite set, there is a one to one correspondence between
elements of the set and Natural numbers
• Example:
• Set S = {2, 3, 4, 5, 6, 7} is a finite set because the number of elements in the set is 6, and 6 is
a natural number.
• Countable sets: N, Z, Q

Slides based on Costas Busch - RPI

 COUNTABLE SETS: EVEN INTEGERS
Example: The set of even integers is countable

Even integers: 0, 2, 4, 6, 

Correspondence:

Positive integers: 1, 2 , 3 , 4 , 

2n corresponds to n 1
Slides based on Costas Busch - RPI
 COUNTABLE SETS: RATIONAL NUMBERS
Example: The set of rational numbers is countable

1 3 7
Rational numbers: , , , 
2 4 8

Slides based on Costas Busch - RPI

 COUNTABLE SETS: RATIONAL NUMBERS
Naïve Proof: 1 1 1
Rational numbers: , , ,
1 2 3
Correspondence:

Positive integers: 1, 2, 3, 
Doesn’t work:
we will never count 2 2 2
numbers with numerator 2: , , ,
1 2 3
Slides based on Costas Busch - RPI
 RATIONAL NUMBERS
Diagonalization to show correspondence with N:
1 1 1 1

1 2 3 4
2 2 2

1 2 3
3 3

1 2
4

1
Slides based on Costas Busch - RPI
 RATIONAL NUMBERS
Diagonalization to show correspondence with N:
1 1 1 1

1 2 3 4
2 2 2

1 2 3
3 3

1 2
4

1
Slides based on Costas Busch - RPI
 RATIONAL NUMBERS
Diagonalization to show correspondence with N:
1 1 1 1

1 2 3 4
2 2 2

1 2 3
3 3

1 2
4

1
Slides based on Costas Busch - RPI
 RATIONAL NUMBERS
Diagonalization to show correspondence with N:
1 1 1 1

1 2 3 4

2 2 2

1 2 3
3 3

1 2
4

Slides based on Costas Busch - RPI
1
 RATIONAL NUMBERS
Diagonalization to show correspondence with N:
1 1 1 1

1 2 3 4

2 2 2

1 2 3
3 3

1 2
4

Slides based on Costas Busch - RPI 1
 RATIONAL NUMBERS
1 1 1 1

1 2 3 4

2 2 2

1 2 3

3 3

1 2
4

Slides based on Costas Busch - RPI
1
 RATIONAL NUMBERS
1 1 2 1 2
Rational Numbers: , , , , , 
1 2 1 3 2

Correspondence:

Positive Integers: 1, 2, 3, 4, 5, 

Slides based on Costas Busch - RPI

 RATIONAL NUMBERS
We have proved that the set of rational numbers is countable by describing
an enumeration procedure.

Slides based on Costas Busch - RPI

 Uncountable Sets

Slides based on Costas Busch - RPI

 UNCOUNTABLE SETS
• Uncountable Sets:
The real numbers are the collection of numbers that can be written out with some kind
of decimal expansion.
– The real numbers include the rational numbers — any fraction of two integers can
be divided out and turned into a decimal. 1/2 = 0.5 and 1/3 = 0.3333..., with
the latter continuing on with 3's forever.
– The real numbers also include irrational numbers, or decimals that go on forever
without settling into a repeated pattern or ending.  is irrational — its decimal
expansion starts out with the familiar 3.14159... but keeps going on forever.

Slides based on Costas Busch - RPI

 UNCOUNTABLE SETS
• Uncountable Sets:
The set of real numbers is an example of an infinite set because it continues to
increase forever without stopping, making it impossible to count its members.
– R is uncountable (“not countable”)

Slides based on Costas Busch - RPI

 UNCOUNTABLE SETS
• Argument for R being Uncountable
R (the set of real numbers) contains infinitely many numbers between any two
numbers.
– Q (the set of rational numbers) has the same property, yet is countable?
– We showed correspondence between the natural numbers and the rationals,
showing that they are all countably infinite and the same size.

Slides based on Costas Busch - RPI

 UNCOUNTABLE SETS
• Argument for R being Uncountable
– The real numbers are an uncountably infinite set — there actually are far more
real numbers than there are natural numbers, and there is no way to line up the
reals and the naturals so that we are assigning exactly one real number to each
natural number.
– An infinite set with infinitely complex numbers should be bigger than N.

Slides based on Costas Busch - RPI

 UNCOUNTABLE SETS
• Proving R being Uncountable
We use proof by contradiction to show prove it.
– Let us suppose that the real numbers are countably infinite, and there is a way to
line up all the reals with the naturals in a one to one correspondence.

Slides based on Costas Busch - RPI

 UNCOUNTABILITY OF R
• Cantor’s Diagonalization Argument:
– Let's say that the first few pairs in the correspondence are the following:

– We assume here is that each and every real number appears somewhere on this
list. We show that this is in fact wrong by making a new number that does not show
up in the list.

Slides based on Costas Busch - RPI

 UNCOUNTABILITY OF R
• For each natural number n, we look at the corresponding real number on
the list, and take the digit n places to the right of the real number's
decimal point. So, take the first digit of the first number, the second digit
of the second number, the third digit of the third number, and so on:

First Natural No. First digit after decimal

Second Natural No. Second digit after decimal
Third Natural No. Third digit after decimal
Fourth Natural No. Fourth digit after decimal
Fifth Natural No. Fifth digit after decimal

Slides based on Costas Busch - RPI

 UNCOUNTABILITY OF R
• From our first real number we get a 5, our second number a 3, and our
third number a 1. We make a new number by taking each of these digits,
and adding 1 to them (flipping around to a 0 if my original digit is 9),
giving us the number 0.64207..., continuing on for all the other numbers
on our list.
• This new "diagonal" number is definitely a real number — it has a
decimal expansion. But it is different from all the numbers on the list: its
first digit is different from the first digit of our first number, its second
digit is different from the second digit of our second number, and so on.

Slides based on Costas Busch - RPI

 UNCOUNTABILITY OF R
• We have made a new real number that does not show up on our list. This
contradicts our main assumption that every real number appears
somewhere in the correspondence.
• Therefore, the set of real numbers is an uncountably infinite set.

Slides based on Costas Busch - RPI

 UNCOUNTABLE SETS
Theorem:
Let S be an infinite countable set

S
The powerset 2 of S is uncountable

Slides based on Costas Busch - RPI

 UNCOUNTABLE SETS
Proof:
Since S is countable, we can write
S  {s1 , s2 , s3 ,}

Elements of S
Elements of the powerset have the form:
{s1 , s3}
{s5 , s7 , s9 , s10 }
……
Slides based on Costas Busch - RPI
 UNCOUNTABLE SETS
We encode each element of the power set with a binary string of 0’s and
1’s Powerset Encoding
element s1 s2 s3 s4 
{s1} 1 0 0 0 

{s2 , s 3 } 0 1 1 0 

{s1 , s 3 , s4 } 1 0 1 1 
Slides based on Costas Busch - RPI
 UNCOUNTABLE SETS
Let’s assume (for contradiction) that the powerset is countable, then we can
enumerate the elements of the powerset.

Slides based on Costas Busch - RPI

 UNCOUNTABLE SETS
Powerset element Encoding
t1 1 0 0 0 0 

t2 1 1 0 0 0 

t3 1 1 0 1 0 

t4 1 1 0 0 1 

Slides based on Costas Busch - RPI
 UNCOUNTABLE SETS
Take the powerset element whose bits are the complements of the diagonal

t1 1 0 0 0 0 
t2 1 1 0 0 0 
t3 1 1 0 1 0 

t4 1 1 0 0 1 
New element: 0011
(birary complement of diagonal)
Slides based on Costas Busch - RPI
 UNCOUNTABLE SETS
The new element must be some ti of the powerset
From the definition of ti the ith bit of ti must be the complement of itself,
which is not possible.

Since we have a contradiction!!!

S
The powerset 2 of S is uncountable

Slides based on Costas Busch - RPI

UNCOUNTABLE SETS: LANGUAGES
Example Alphabet : {a, b}
The set of all Strings:
*
S  {a, b}  { , a, b, aa, ab, ba, bb, aaa , aab,}
A language is a subset of S

L  {aa, ab, aab, ...} infinite and countable

The powerset of S contains all languages:

S
2  {{},{a},{a, b}{aa, ab, aab},}
L1 L2 L3 L4 
Uncountable
 UNCOUNTABLE SETS: LANGUAGES
Languages: uncountable
L1 L2 L3  Lk 

M1 M2 M3 ?
Turing machines: countable

But, there are more languages than Turing Machines

Slides based on Costas Busch - RPI

 UNCOUNTABLE SETS: LANGUAGES
Languages not accepted by Turing Machines

Lk

Languages
Accepted by
Turing Machines

Slides based on Costas Busch - RPI