Theory of computation

Lecture 5: Automata and Regular languages

Adnane Saoud

Tuesday, October 17, 2023

1
 Summary of lecture 4

Lecture 4
§ Context free languages (CFLs)
§ Pushdown automata (PDA)
§ Converting CFGs to PDAs

Lecture 5
§ Proving languages not Context Free
§ Turing machines

2
Pumping Lemma for
Context-free
Languages

3
 Proving languages not Context Free

Recall Theorem: 𝐴 is a CFL iff some PDA recognizes 𝐴

Let 𝐵 = 0! 1! 2! 𝑘 ≥ 0}. We will show that 𝐵 isn’t a CFL.
Pumping Lemma for CFLs: For every CFL 𝐴, there is a 𝑝
such that if 𝑠 ∈ 𝐴 and 𝑠 ≥ 𝑝 then there exists u,v,x,y,z
with 𝑠 = 𝑢𝑣𝑥𝑦𝑧 where
1) 𝑢𝑣 " 𝑥𝑦 " 𝑧 ∈ 𝐴 for all 𝑖 ≥ 0
2) 𝑣𝑦 ≠ ε
3) 𝑣𝑥𝑦 ≤ 𝑝

Informally: All long strings in 𝐴 are pumpable and stay in 𝐴.

≥𝑝
𝑢 𝑣 𝑥 𝑦 𝑧
𝑠= ∈𝐴

∈𝐴
𝑢 𝑣 𝑣 𝑥 𝑦 𝑦 𝑧
≤𝑝 4
 Pumping Lemma – Sketch of Proof
Pumping Lemma for CFLs: For every CFL 𝐴, there is a 𝑝
such that if 𝑠 ∈ 𝐴 and 𝑠 ≥ 𝑝 then 𝑠 = 𝑢𝑣𝑥𝑦𝑧 where
1) 𝑢𝑣 ! 𝑥𝑦 ! 𝑧 ∈ 𝐴 for all 𝑖 ≥ 0
2) 𝑣𝑦 ≠ ε
3) 𝑣𝑥𝑦 ≤ 𝑝

Proof by picture: E

𝑠= 𝑣 𝑦 𝑧
𝑢 𝑥 Long 𝑠 →
tall parse tree

5
 Pumping Lemma – Sketch of Proof
Pumping Lemma for CFLs: For every CFL 𝐴, there is a 𝑝
such that if 𝑠 ∈ 𝐴 and 𝑠 ≥ 𝑝 then 𝑠 = 𝑢𝑣𝑥𝑦𝑧 where
1) 𝑢𝑣 ! 𝑥𝑦 ! 𝑧 ∈ 𝐴 for all 𝑖 ≥ 0
2) 𝑣𝑦 ≠ ε
3) 𝑣𝑥𝑦 ≤ 𝑝

Proof by picture: E

R
E
R
𝑢 𝑣 𝑦 𝑧 R
R
𝑥
𝑣 𝑥 𝑦
Generates 𝑢𝑣𝑣𝑥𝑦𝑦𝑧 𝑢 𝑧
Generates 𝑢𝑥𝑧
= 𝑢𝑣 ! 𝑥𝑦 ! 𝑧
= 𝑢𝑣 " 𝑥𝑦 " 𝑧

“cutting and pasting” argument 6

 Pumping Lemma – Sketch of Proof
Let 𝑏 = the length of the longest right hand side of a rule (E → E+T) E
Let ℎ = the height of the parse tree for 𝑠. E+T
A tree of height ℎ and max branching 𝑏 has at most 𝑏 " leaves (So 𝑠 ≤ 𝑏 " ).
E

want
R ℎ> 𝑉

𝑠= 𝑢 𝑣 𝑥 𝑦 𝑧
use 𝑠 > 𝑏 # set 𝑝 = 𝑏 # + 1

Let 𝑝 = 𝑏 # + 1 where 𝑉 = # variables in the grammar. So if 𝑠 ≥ 𝑝 > 𝑏 # then 𝑠 >

𝑏 |#| and so ℎ > 𝑉 .
Thus at least 𝑉 + 1 variables occur in the longest path. So some variable 𝑅 must repeat
on a path.
 Pumping Lemma: main idea

For 𝑠 ∈ 𝐴 where 𝑠 ≥ 𝑝, we have 𝑠 = 𝑢𝑣𝑥𝑦𝑧 where:

1) 𝑢𝑣 ! 𝑥𝑦 ! 𝑧 ∈ 𝐴 for all 𝑖 ≥ 0 …cutting and pasting
2) 𝑣𝑦 ≠ ε
3) 𝑣𝑥𝑦 ≤ 𝑝

want
R ℎ> 𝑉

𝑠= 𝑢 𝑣 𝑥 𝑦 𝑧
use 𝑠 > 𝑏 #
set 𝑝 = 𝑏 # + 1 8
 Example 1 of Proving Non-CF
Pumping Lemma for CFLs: For every CFL 𝐴, there is a 𝑝
such that if 𝑠 ∈ 𝐴 and 𝑠 ≥ 𝑝 then 𝑠 = 𝑢𝑣𝑥𝑦𝑧 where
1) 𝑢𝑣 ! 𝑥𝑦 ! 𝑧 ∈ 𝐴 for all 𝑖 ≥ 0
2) 𝑣𝑦 ≠ ε
3) 𝑣𝑥𝑦 ≤ 𝑝

Let 𝐵 = 0! 1! 2! 𝑘 ≥ 0}
Show: 𝐵 is not a CFL 𝑠 = 00 ⋯ 0011 ⋯ 1122 ⋯ 22
Proof by Contradiction: 𝑢 𝑣 𝑥 𝑦 𝑧
≤𝑝
Assume (to get a contradiction) that 𝐵 is a CFL .
The CFL pumping lemma gives 𝑝 as above. Let 𝑠 = 0$ 1$ 2$ ∈ 𝐵.
Pumping lemma says that can divide 𝑠 = 𝑢𝑣𝑥𝑦𝑧 satisfying the 3 conditions.
Condition 3 ( 𝑣𝑥𝑦 ≤ 𝑝) implies that 𝑣𝑥𝑦 cannot contain both 0s and 2s.
So 𝑢𝑣 % 𝑥𝑦 % 𝑧 has unequal numbers of 0s, 1s, and 2s.
Thus 𝑢𝑣 % 𝑥𝑦 % 𝑧 ∉ 𝐵, violating Condition 1. Contradiction!
Therefore our assumption (𝐵 is a CFL) is false. We conclude that 𝐵 is not a CFL .
9
 Example 2 of Proving Non-CF
Pumping Lemma for CFLs: For every CFL 𝐴, there is a 𝑝
such that if 𝑠 ∈ 𝐴 and 𝑠 ≥ 𝑝 then 𝑠 = 𝑢𝑣𝑥𝑦𝑧 where
1) 𝑢𝑣 ! 𝑥𝑦 ! 𝑧 ∈ 𝐴 for all 𝑖 ≥ 0
2) 𝑣𝑦 ≠ ε
3) 𝑣𝑥𝑦 ≤ 𝑝

Let 𝐹 = 𝑤𝑤 𝑤 ∈ Σ ∗ } . Σ = {0,1}.
Show: 𝐹 is not a CFL.
𝑠% = 0 ⋯ 01 ⋯ 10 ⋯ 01 ⋯ 1
Assume (for contradiction) that 𝐹 is a CFL. 𝑢 𝑣 𝑥 𝑦 𝑧
The CFL pumping lemma gives 𝑝 as above.
Need to choose 𝑠 ∈ 𝐹. Which 𝑠?
Try 𝑠% = 0$ 1$ 0$ 1$ ∈ 𝐹.
Show 𝑠% cannot be pumped 𝑠% = 𝑢𝑣𝑥𝑦𝑧 satisfying the 3 conditions.
Condition 3 implies that 𝑣𝑥𝑦 does not overlap two runs of 0s or two runs of 1s.
Therefore, in 𝑢𝑣 % 𝑥𝑦 % 𝑧, two runs of 0s or two runs of 1s have unequal length.
So 𝑢𝑣 % 𝑥𝑦 % 𝑧 ∉ 𝐹 violating Condition 1. Contradiction! Thus 𝐹 is not a CFL.

10
 CFG are not Closed Under Union

Let 𝐴% = 0& 1& 2' 𝑘, 𝑙 ≥ 0} (equal #s of 0s and 1s)

Let 𝐴! = 0' 1& 2& 𝑘, 𝑙 ≥ 0} (equal #s of 1s and 2s)
Observe that PDAs can recognize 𝐴% and 𝐴! . What can we now
conclude?
a) The class of CFLs is not closed under intersection.
b) The Pumping Lemma shows that 𝐴% ∪ 𝐴! is not a CFL.
c) The class of CFLs is closed under complement.

11
Turing machines

12
 Turing Machines (TMs)

head
a b a b b ˽ ˽ ...
Finite
control read/write input tape

1) Head can read and write

2) Head is two way (can move left or right)
3) Tape is infinite (to the right)
4) Infinitely many blanks “˽“ follow input
5) Can accept or reject any time (not only at end of input)

13
 TM – example

TM recognizing 𝐵 = a! b! c! 𝑘 ≥ 0
1) Scan right until ˽ while checking if input is in a∗b∗c∗, reject if not.
2) Return head to left end.
3) Scan right, crossing off single a, b, and c.
4) If the last one of each symbol, accept.
5) If the last one of some symbol but not others, reject.
6) If all symbols remain, return to left end and repeat from (3).
head
input tape
Finite a a a b b b c c c ˽ ˽
control

accept

14
 TM – example

Exercise:
How do we get the effect of “crossing off” with a Turing machine?
a) We add that feature to the model.
b) We use a tape alphabet Γ = {a, b, c, a, b, c, ˽ }.
c) All Turing machines come with an eraser.

15
 TM – Formal Definition

Definition: A Turing Machine (TM) is a 7-tuple (𝑄, Σ, Γ, 𝛿, 𝑞@, 𝑞acc , 𝑞rej)

Σ input alphabet
Γ tape alphabet (Σ ⊆ Γ)
𝛿: Q×Γ → 𝑄×Γ× {L, R} (L = Left, R = Right)
𝛿 𝑞, a = (𝑟, b, R)

On input 𝑤 a TM 𝑀 may halt (enter 𝑞acc or 𝑞rej)

or 𝑀 may run forever (“loop”).

So 𝑀 has 3 possible outcomes for each input 𝑤:

1. Accept 𝑤 (enter 𝑞acc )
2. Reject 𝑤 by halting (enter 𝑞rej )
3. Reject 𝑤 by looping (running forever)

16
 TM Recognizers and Deciders

Let 𝑀 be a TM. Then 𝐿 𝑀 = 𝑤 𝑀

accepts 𝑤}.
Say that 𝑀 recognizes 𝐴 if 𝐴 = 𝐿(𝑀).
Definition: 𝐴 is Turing-recognizable if T-recognizable
𝐴 = 𝐿(𝑀) for some TM 𝑀.

Definition: TM 𝑀 is a decider if 𝑀 halts

T-decidable
on all inputs.
CFLs
Say that 𝑀 decides 𝐴 if 𝐴 = 𝐿(𝑀) and
𝑀 is a decider.
regular
Definition: 𝐴 is Turing-decidable if 𝐴 =
𝐿(𝑀) for some TM decider 𝑀.
17
 Multi-tape Turing machines

input tape
Finite
control
} work tapes, initially blank

...
all tapes read/write

Theorem: 𝐴 is T-recognizable iff some multi-tape TM recognizes 𝐴

Proof: (→) immediate. (←) convert multi-tape to single tape:

18
 Multi-tape Turing machines

a a b b a ˽ ˽ ...
multi-tape 𝑀 1 0 1 ˽ ...

...
c c c a ˽ ...

single tape 𝑆 a a b b a # 1 0 1 # … # c c c a ˽ ˽

𝑆 simulates 𝑀 by storing the contents of multiple tapes on a single tape in “blocks”.

Record head positions with dotted symbols.
Some details of 𝑆:
1) To simulate each of 𝑀’s steps
a. Scan entire tape to find dotted symbols.
b. Scan again to update according to 𝑀’s 𝛿.
c. Shift to add room as needed.
2) Accept/reject if 𝑀 does.

19
Summary

20
 Summary

1. Proved the CFL Pumping Lemma as a tool for showing that

languages are not context free.
2. Defined Turing machines (TMs).
3. Defined TM deciders (halt on all inputs).
4. T-recognizable and T-decidable languages.

21