15-453

FORMAL LANGUAGES,
AUTOMATA AND
COMPUTABILITY
 15-453
FORMAL LANGUAGES,
AUTOMATA AND
COMPUTABILITY
YOU NEED TO PICK UP
• THE SYLLABUS,
• THE COURSE SCHEDULE,
• THE PROJECT INFO SHEET,
• TODAY’S CLASS NOTES
WWW.FLAC.WS
 INSTRUCTORS & TAs

Lenore Blum Aashish Jindia Andy Smith

 Office Hours

Lenore Blum (lblum@cs)

Office Hours: Tues 2-3pm, Gates 7105
Andy Smith (adsmith@andrew)
Office Hours: Mon 6-8pm, GHC 7101

Aashish Jindia (ajindia@andrew)

Office Hours: Wed 6-8pm, GHC 7101
 Project

Homework 15% Class

Participation
25%
5+5%
Midterm I

Final 15%
25% Midterm II

15%
 Project

Homework 15% Class

Participation
25%
5+5%
Midterm I

Final 15%
25% Midterm II

15%
 HOMEWORK
Homework will be assigned every Tuesday and
will be due one week later at the beginning of
class. Late homework will be accepted only under
exceptional circumstances.

All assignments must be typeset (exceptions

allowed for diagrams). Each problem should be
done on a separate page.

You must list your collaborators (including

yourself) and all references (including books,
articles, websites, people) in every homework
assignment in a References section at the end.
 COURSE PROJECT

Choose a (unique) topic

Learn about your topic

Write progress reports
(Feb 6, March 22)

Meet with an instructor/TA once a

month
Prepare a 10-minute presentation
(April 24, April 29, May 1?)

Final Report (May 1)

 COURSE PROJECT
Suggested places to look for project topics

Any paper that has appeared in the proceedings of FOCS or

STOC in the last 5 years. FOCS (Foundations of Computer
Science) and STOC (Symposium on the Theory of Computing)
are the two major conferences of general computer science
theory. The proceedings of both conferences are available at
the E&S library or electronically.

· Electronic version of the proceedings of STOC

· Electronic version of the proceedings of FOCS

• What's New]
This class is about mathematical
models of computation
 WHY SHOULD I CARE?

WAYS OF THINKING

THEORY CAN DRIVE PRACTICE

Mathematical models of computation
predated computers as we know them

THIS STUFF IS USEFUL

 Course Outline
PART 1
Automata and Languages:
finite automata, regular languages, pushdown automata, context-free
languages, pumping lemmas.

PART 2
Computability Theory:
Turing Machines, decidability, reducibility, the arithmetic hierarchy, the
recursion theorem, the Post correspondence problem.

PART 3
Complexity Theory and Applications:
time complexity, classes P and NP, NP-completeness, space complexity,
PSPACE, PSPACE-completeness, the polynomial hierarchy, randomized
complexity, classes RP and BPP.
Mathematical Models of Computation
(predated computers as we know them)
PART 1 1940’s-50’s (neurophysiology,
Automata and Languages: linguistics)
finite automata, regular languages, pushdown automata, context-free
languages, pumping lemmas.

PART 2
Computability Theory: 1930’s-40’s (logic, decidability)
Turing Machines, decidability, reducibility, the arithmetic hierarchy, the
recursion theorem, the Post correspondence problem.

PART 3 1960’s-70’s
Complexity Theory and Applications: (computers)
time complexity, classes P and NP, NP-completeness, space complexity,
PSPACE, PSPACE-completeness, the polynomial hierarchy, randomized
complexity, classes RP and BPP.
This class will emphasize PROOFS

A good proof should be:

Easy to understand
Correct
Suppose A ⊆ {1, 2, …, 2n} with |A| = n+1

TRUE or FALSE:
There are always two numbers in A
such that one divides the other

TRUE
LEVEL 1

HINT 1:
THE PIGEONHOLE PRINCIPLE
If you put 6 pigeons in 5 holes
then at least one hole will have
more than one pigeon
LEVEL 1

HINT 1:
THE PIGEONHOLE PRINCIPLE
If you put 6 pigeons in 5 holes
then at least one hole will have
more than one pigeon
LEVEL 1

HINT 1:
THE PIGEONHOLE PRINCIPLE
If you put n+1 pigeons in n
holes then at least one hole will
have more than one pigeon

HINT 2:
Every integer a can be written as
a = 2km, where m is an odd number
LEVEL 2
PROOF IDEA:
Given: A ⊆ {1, 2, …, 2n} and |A| = n+1

Show: There is an integer m and elements

a1 = a2 in A
such that a1 = 2im and a2 = 2jm
LEVEL 3
PROOF:
Suppose A ⊆ {1, 2, …, 2n} with |A| = n+1

Write every number in A as a = 2km, where

m is an odd number between 1 and 2n-1

How many odd numbers in {1, …, 2n}? n

Since |A| = n+1, there must be two numbers

in A with the same odd part

Say a1 and a2 have the same odd part m.

Then a1 = 2im and a2 = 2jm, so one must
divide the other
 We expect your proofs to have three levels:

 The first level should be a one-word or

one-phrase “HINT” of the proof

(e.g. “Proof by contradiction,” “Proof by induction,”

“Follows from the pigeonhole principle”)

 The second level should be a short one-

paragraph description or “KEY IDEA”

 The third level should be the FULL PROOF

 DOUBLE STANDARDS?

During the lectures, my proofs will usually

only contain the first two levels and maybe
part of the third
DETERMINISTIC FINITE
AUTOMATA
DETERMINISTIC FINITE
AUTOMATA
and
REGULAR LANGUAGES

TUESDAY JAN 14
 11 1
0
0,1
1
0111 111 1
0 0

Read string left to right 1

The automaton accepts a string if the

process ends in a double circle
 ANATOMY OF A DETERMINISTIC
FINITE AUTOMATON
states accept states (F)
q1 1
0
0,1
1
q0 q2

0 0

1
start state (q0) q3 states
 SOME VOCABULARY
An alphabet Σ is a finite set (e.g., Σ = {0,1})

A string over Σ is a finite-length sequence of

elements of Σ

Σ* = the set of strings over Σ

For string x, |x| is the length of x

The unique string of length 0 will be denoted

by ε and will be called the empty or null string
A language over Σ is a set of strings over Σ
In other words: a language is a subset of Σ*
 deterministic DFA
A ^ finite automaton ^ is a 5-tuple M = (Q, Σ, δ, q0, F)
Q is the set of states (finite)
Σ is the alphabet (finite)
δ : Q × Σ → Q is the transition function
q0 ∈ Q is the start state
F ⊆ Q is the set of accept/final states

Suppose w1, ... , wn ∈ Σ and w = w1... wn ∈ Σ*

Then M accepts w iff there are r0, r1, ..., rn ∈ Q, s.t.
• r0 = q0
• δ(ri, wi+1) = ri+1, for i = 0, ..., n-1, and
• rn ∈ F
 deterministic DFA
A ^ finite automaton ^ is a 5-tuple M = (Q, Σ, δ, q0, F)
Q is the set of states (finite)
Σ is the alphabet (finite)
δ : Q × Σ → Q is the transition function
q0 ∈ Q is the start state
F ⊆ Q is the set of accept/final states

M accepts ε iff q0 ∈ F

L(M) = set of all strings that M accepts

= “the language recognized by M”
 q0 0,1

L(M) = ?
 q0 0,1

L(M) = ?
0 0

1
q0 q1
1

L(M) = ?
 0 0

1
q0 q1
1

L(M) = { w | w has an even number of 1s}

 Q Σ q0 F
δ 0 1
M = ({p,q}, {0,1}, δ, p, {q})
p p q
0 0
q q p

1
p q
1
 Q Σ q0 F
δ 0 1
M = ({p,q}, {0,1}, δ, p, {q})
p p q
0 0
q q p

1 THEOREM:
p q L(M) = {w | w has odd
1 number of 1s }
 Q Σ q0 F
δ 0 1
M = ({p,q}, {0,1}, δ, p, {q})
p p q
0 0
q q p

1 THEOREM:
p q L(M) = {w | w has odd
1 number of 1s }
Proof: By induction on n, the length of a string.
Base Case: n=0: ε ∉ RHS and ε ∉ L(M). Why?
 Q Σ q0 F
δ 0 1
M = ({p,q}, {0,1}, δ, p, {q})
p p q
0 0
q q p

1 THEOREM:
p q L(M) = {w | w has odd
1 number of 1s }
Proof: By induction on n, the length of a string.
Base Case: n=0: ε ∉ RHS and ε ∉ L(M). Why?
Induction Hypothesis: Suppose for all w ∈ Σ*, |w| = n,
w ∈ L(M) iff w has odd number of 1s.
Induction step: Any string of length n+1 has the form w0 or w1.
 Q Σ q0 F
δ 0 1
M = ({p,q}, {0,1}, δ, p, {q})
p p q
0 0
q q p

1 THEOREM:
p q L(M) = {w | w has odd
1 number of 1s }
Proof: By induction on n, the length of a string.
Base Case: n=0: ε ∉ RHS and ε ∉ L(M). Why?
Induction Hypothesis: Suppose for all w ∈ Σ*, |w| = n,
w ∈ L(M) iff w has odd number of 1s.
Induction step: Any string of length n+1 has the form w0 or w1.
Now w0 has an odd # of 1’s ⇔ w has an odd # of 1’s⇔
M is in state q after reading w (why?) ⇔
M is in state q after reading w0 (why?) ⇔w0 ∈ L(M)
 Q Σ q0 F
δ 0 1
M = ({p,q}, {0,1}, δ, p, {q})
p p q
0 0
q q p

1 THEOREM:
p q L(M) = {w | w has odd
1 number of 1s }
Proof: By induction on n, the length of a string.
Base Case: n=0: ε ∉ RHS and ε ∉ L(M). Why?
Induction Hypothesis: Suppose for all w ∈ Σ*, |w| = n,
w ∈ L(M) iff w has odd number of 1s.
Induction step: Any string of length n+1 has the form w0 or w1.
Now w1 has an odd # of 1’s ⇔ w has an even # of 1’s⇔
M is in state p after reading w (why?) ⇔
M is in state q after reading w1 (why?) ⇔w1 ∈ L(M) QED
Build a DFA that accepts all and only those
strings that contain 001
Build a DFA that accepts all and only those
strings that contain 001

0,1
1 0

0 0 1
q q0 q00 q001
1
 DEFINITION: A language L is
regular if it is recognized by a DFA,
i.e. if there is a DFA M s.t. L = L(M).

L = { w | w contains 001} is regular

L = { w | w has an even number of 1s} is regular

 UNION THEOREM
Given two languages, L1 and L2, define
the union of L1 and L2 as
L1 ∪ L2 = { w | w ∈ L1 or w ∈ L2 }

Theorem: The union of two regular

languages is also a regular language
 Theorem: The union of two regular
languages is also a regular language

Proof: Let
M1 = (Q1, Σ, δ1, q01, F1) be finite automaton for L1
and
M2 = (Q2, Σ, δ2, q0, F2) be finite automaton for L2
2

We want to construct a finite automaton

M = (Q, Σ, δ, q0, F) that recognizes L = L1 ∪ L2
 Idea: Run both M1 and M2 at the same time!

Q = pairs of states, one from M1 and one from M2

= { (q1, q2) | q1 ∈ Q1 and q2 ∈ Q2 }
= Q1 × Q2

q0 = (q10, q20)

F = { (q1, q2) | q1 ∈ F1 or q2 ∈ F2 }

δ( (q1,q2), σ) = (δ1(q1, σ), δ2(q2, σ))

 Intersection THEOREM
Given two languages, L1 and L2, define
the intersection of L1 and L2 as
L1 ∩ L2 = { w | w ∈ L1 and w ∈ L2 }

Theorem: The intersection of two

regular languages is also a regular
language
0
0

1
q0 q1
1 1
1

0
p0 p1
0

UNION? INTERSECTION?
 UNION

1
q0,p0 q1,p0
1

0 0
0 0

1
q0,p1 q1,p1
1
 INTERSECTION

1
q0,p0 q1,p0
1

0 0
0 0

1
q0,p1 q1,p1
1
 Intersection THEOREM
Given two languages, L1 and L2, define
the intersection of L1 and L2 as
L1 ∩ L2 = { w | w ∈ L1 and w ∈ L2 }

Theorem: The intersection of two

regular languages is also a regular
language
 Show:

L = { x {1, 2, 3}* | the digits 1, 2 and 3

appear in x in that order, but not
necessarily consecutively}
is regular.
 Show:

L = { x {0,1}* | x≠ ε and d(x) ≡ 0 mod 3 }

is regular.

(d(x) is the natural # corresponding to x.)

 WWW.FLAC.WS
YOU NEED TO PICK UP
THE SYLLABUS,
THE COURSE SCHEDULE,
THE PROJECT INFO SHEET,
TODAY’S CLASS NOTES

Read Chapters 0, 1.1 and 1.2 of Sipser for next time,

Also Rabin-Scott paper