Other Models of
Computation
1
Models of computation:
•Turing Machines
•Recursive Functions
•Post Systems
•Rewriting Systems
2
Church’s Thesis:
All models of computation are equivalent
Turing’s Thesis:
A computation is mechanical if and only if
it can be performed by a Turing Machine
3
Church’s and Turing’s Thesis are similar:
Church-Turing Thesis
4
Recursive Functions
An example function:
2
f ( n) n 1
Domain Range
3 f (3) 10 10
5
We need a way to define functions
We need a set of basic functions
6
Basic Primitive Recursive Functions
Zero function: z ( x) 0
Successor function: s ( x) x 1
Projection functions: p1( x1, x2 ) x1
p2 ( x1, x2 ) x2
7
Building complicated functions:
Composition: f ( x, y ) h( g1( x, y ), g 2 ( x, y ))
Primitive Recursion:
f ( x,0) g1( x)
f ( x, y 1) h( g 2 ( x, y ), f ( x, y ))
8
Any function built from
the basic primitive recursive functions
is called:
Primitive Recursive Function
9
A Primitive Recursive Function: add ( x, y )
add ( x,0) x (projection)
add ( x, y 1) s (add ( x, y ))
(successor function)
10
add (3,2) s (add (3,1))
s ( s (add (3,0)))
s ( s (3))
s ( 4)
5
11
Another Primitive Recursive Function:
mult ( x, y )
mult ( x,0) 0
mult ( x, y 1) add ( x, mult ( x, y ))
12
Theorem:
The set of primitive recursive functions
is countable
Proof:
Each primitive recursive function
can be encoded as a string
Enumerate all strings in proper order
Check if a string is a function
13
Theorem
there is a function that
is not primitive recursive
Proof:
Enumerate the primitive recursive functions
f1, f 2 , f3 ,
14
Define function g (i ) fi (i ) 1
g differs from every fi
g is not primitive recursive
END OF PROOF 15
A specific function that is not
Primitive Recursive:
Ackermann’s function:
A(0, y ) y 1
A( x,0) A( x 1,1)
A( x, y 1) A( x 1, A( x, y ))
Grows very fast,
faster than any primitive recursive function
16
Recursive Functions
y ( g ( x, y )) smallest y such that g ( x, y ) 0
Accerman’s function is a
Recursive Function
17
Recursive Functions
Primitive recursive functions
18
Post Systems
• Have Axioms
• Have Productions
Very similar with unrestricted grammars
19
Example: Unary Addition
Axiom: 1 1 11
Productions:
V1 V2 V3 V11 V2 V31
V1 V2 V3 V1 V21 V31
20
A production:
V1 V2 V3 V11 V2 V31
1 1 11 11 1 111 11 11 1111
V1 V2 V3 V1 V21 V31
21
Post systems are good for
proving mathematical statements
from a set of Axioms
22
Theorem:
A language is recursively enumerable
if and only if
a Post system generates it
23
Rewriting Systems
They convert one string to another
• Matrix Grammars
• Markov Algorithms
• Lindenmayer-Systems
Very similar to unrestricted grammars
24
Matrix Grammars
Example: P1 : S S1S2
P2 : S1 aS1, S2 bS2c
P3 : S1 , S2
Derivation:
S S1S2 aS1bS2c aaS1bbS2cc aabbcc
A set of productions is applied simultaneously
25
P1 : S S1S2
P2 : S1 aS1, S2 bS2c
P3 : S1 , S 2
n n n
L {a b c : n 0}
Theorem:
A language is recursively enumerable
if and only if
a Matrix grammar generates it 26
Markov Algorithms
Grammars that produce
Example: ab S
aSb S
S .
Derivation:
aaabbb aaSbb aSb S
27
ab S
aSb S
S .
n n
L {a b : n 0}
28
*
In general: L {w : w }
Theorem:
A language is recursively enumerable
if and only if
a Markov algorithm generates it
29
Lindenmayer-Systems
They are parallel rewriting systems
Example: a aa
Derivation: a aa aaaa aaaaaaaa
n
2
L {a : n 0}
30
Lindenmayer-Systems are not general
As recursively enumerable languages
Extended Lindenmayer-Systems: ( x, a, y ) u
context
Theorem:
A language is recursively enumerable
if and only if an
Extended Lindenmayer-System generates it
31
Computational Complexity
32
Time Complexity: The number of steps
during a computation
Space Complexity: Space used
during a computation
33
Time Complexity
•We use a multitape Turing machine
•We count the number of steps until
a string is accepted
•We use the O(k) notation
34
Example: n n
L {a b : n 0}
Algorithm to accept a string w :
•Use a two-tape Turing machine
•Copy the a on the second tape
a
•Compare the and b
35
n n
L {a b : n 0}
Time needed:
•Copy the a on the second tape O (| w |)
•Compare the a and b O (| w |)
Total time: O (| w |)
36
n n
L {a b : n 0}
For string of length n
time needed for acceptance: O (n)
37
Language class: DTIME (n)
DTIME (n)
L1 L3
L2
A Deterministic Turing Machine
accepts each string of length n
in time O (n)
38
DTIME (n)
n n
{a b : n 0}
{ww}
39
In a similar way we define the class
DTIME (T (n))
for any time function: T (n)
2 3
Examples: DTIME (n ), DTIME (n ),...
40
Example: The membership problem
for context free languages
L {w : w is generated by grammar G}
3
L DTIME (n ) (CYK - algorithm)
Polynomial time
41
k 1 k
Theorem: DTIME (n ) DTIME (n )
k 1
DTIME (n )
k
DTIME (n )
42
k
Polynomial time algorithms: DTIME (n )
Represent tractable algorithms:
For small k we can compute the
result fast
43
The class P
k
P DTIME (n ) for all k
•Polynomial time
•All tractable problems
44
P
CYK-algorithm n n
{a b }
{ww}
45
n
Exponential time algorithms: DTIME (2 )
Represent intractable algorithms:
Some problem instances
may take centuries to solve
46
Example: the Traveling Salesperson Problem
5
3
4 1 2
2
6
10
8
3
Question: what is the shortest route that
connects all cities? 47
5
3
4 1 2
2
6
10
8
3
Question: what is the shortest route that
connects all cities? 48
A solution: search exhuastively all
hamiltonian paths
L = {shortest hamiltonian paths}
n
L DTIME (n!) DTIME (2 )
Exponential time
Intractable problem
49
Example: The Satisfiability Problem
Boolean expressions in
Conjunctive Normal Form:
t1 t2 t3 tk
ti x1 x2 x3 x p
Variables
Question: is expression satisfiable?
50
Example: ( x1 x2 ) ( x1 x3 )
Satisfiable: x1 0, x2 1, x3 1
( x1 x2 ) ( x1 x3 ) 1
51
Example: ( x1 x2 ) x1 x2
Not satisfiable
52
L {w : expression w is satisfiable}
n
For n variables: L DTIME (2 )
exponential
Algorithm:
search exhaustively all the possible
binary values of the variables
53
Non-Determinism
Language class: NTIME (n)
NTIME (n)
L1 L3
L2
A Non-Deterministic Turing Machine
accepts each string of length n
in time O (n)
54
Example: L {ww}
Non-Deterministic Algorithm
to accept a string ww :
•Use a two-tape Turing machine
•Guess the middle of the string
and copy w on the second tape
•Compare the two tapes 55
L {ww}
Time needed:
•Use a two-tape Turing machine
•Guess the middle of the string O (| w |)
and copy w on the second tape
O (| w |)
•Compare the two tapes
Total time: O (| w |)
56
NTIME (n)
L {ww}
57
In a similar way we define the class
NTIME (T (n))
for any time function: T (n)
2 3
Examples: NTIME (n ), NTIME (n ),...
58
Non-Deterministic Polynomial time algorithms:
k
L NTIME ( n )
59
The class NP
k
P NTIME (n ) for all k
Non-Deterministic Polynomial time
60
Example: The satisfiability problem
L {w : expression w is satisfiable}
Non-Deterministic algorithm:
•Guess an assignment of the variables
•Check if this is a satisfying assignment
61
L {w : expression w is satisfiable}
Time for n variables:
•Guess an assignment of the variables O (n)
•Check if this is a satisfying assignment O (n)
Total time: O (n)
62
L {w : expression w is satisfiable}
L NP
The satisfiability problem is an NP - Problem
63
Observation:
P NP
Deterministic Non-Deterministic
Polynomial Polynomial
64
Open Problem: P NP ?
WE DO NOT KNOW THE ANSWER
65
Open Problem: P NP ?
Example: Does the Satisfiability problem
have a polynomial time
deterministic algorithm?
WE DO NOT KNOW THE ANSWER
66
NP-Completeness
A problem is NP-complete if:
•It is in NP
•Every NP problem is reduced to it
(in polynomial time)
67
Observation:
If we can solve any NP-complete problem
in Deterministic Polynomial Time (P time)
then we know:
P NP
68
Observation:
If we prove that
we cannot solve an NP-complete problem
in Deterministic Polynomial Time (P time)
then we know:
P NP
69
Cook’s Theorem:
The satisfiability problem is NP-complete
Proof:
Convert a Non-Deterministic Turing Machine
to a Boolean expression
in conjunctive normal form
70
Other NP-Complete Problems:
•The Traveling Salesperson Problem
•Vertex cover
•Hamiltonian Path
All the above are reduced
to the satisfiability problem
71
Observations:
It is unlikely that NP-complete
problems are in P
The NP-complete problems have
exponential time algorithms
Approximations of these problems
are in P
72
