Regular and Nonregular

Languages

Chapter 8 1
 Languages: Regular or Not?
a*b* is regular.
{anbn: n  0} is not.

{w  {a, b}* : every a is immediately followed by b} is

regular.

{w  {a, b}* : every a has a matching b somewhere} is

not

● Showing that a language is regular.

● Showing that a language is not regular.

2
 How Many Regular Languages?
Theorem: There is a countably infinite number of regular
languages.

Proof:
● Upper bound on number of regular languages:
number of FSMs (or regular expressions).

● Lower bound on number of regular languages:

{a},{aa},{aaa},{aaaa},{aaaaa},{aaaaaa},…

are all regular. That set is countably infinite.

3
 Regular And Nonregular
Languages?
There is a countably infinite number of regular languages.

There is an uncountably infinite number of languages over

any nonempty alphabet .

So there are many more nonregular languages than there

are regular ones.

4
Showing that a Language is Regular
Theorem: Every finite language is regular.

Proof: If L is the empty set, then it is defined by the

regular expression  and so is regular. If it is any finite
language composed of the strings s1, s2, … sn for some
positive integer n, then it is defined by the regular
expression:

s1  s2  …  sn

So it too is regular.

5
Showing that a Language is Regular

Example:

Let L = L1  L2, where:

L1 = {anbn, n  0}, and
L2 = {bnan, n  0}

L=

Is L1 regular?
Is L2 regular?
Is L regular?

6
Finiteness - Theoretical vs. Practical
Any finite language is regular. The size of the language doesn't
matter.

Parity Soc. Sec. #

Checking Checking

But, from an implementation point of view, it very well may.

When is an FSM a good way to encode the facts about a

language?

FSM’s are good at looking for repeating patterns. They don't

bring much to the table when the language is just a set of
unrelated strings.

7
Regular Does Not Always Mean Tractable

Let  = {12, 13, 21, 23, 31, 32}.

Let L be the language of strings that correspond to

successful move sequences. The shortest string in L
has length 264 -1.

There is an FSM that accepts L:

8
 Showing That L is Regular
1. Show that L is finite.

2. Exhibit an FSM for L.

3. Exhibit a regular expression for L.

4. Show that the number of equivalence classes of L is

finite.

5. Exhibit a regular grammar for L.

6. Exploit the closure theorems.

9
Closure Properties of Regular Languages
● Union

Proved by same constructions used

● Concatenation
in the proof of Kleene’s Theorem

● Kleene star

● Complement

● Intersection

● Difference

● Reverse

● Letter substitution 10
Closure of Regular Languages
Under Complement

11
 Closure of Regular Languages
Under Complement
What about:

On input bbb?

12
 Closure of Regular Languages
Under Complement
What about:

On input a?

13
A Complement Example

14
 The Complement Procedure
Given: an FSM M,
Construct a new machine to accept L(M):

• If necessary, use ndfsmtodfsm to construct M, a

deterministic equivalent of M.

• Make sure that M is described completely.

• Swap accepting and nonaccepting states.

15
 Closure of Regular Languages
Under Intersection
L1  L2 =

L1 L2

Write this in terms of operations we have already proved

closure for:

● Union
● Concatenation
● Kleenestar
● Complementation
16
 Closure of Regular Languages
Under Intersection
L1  L2 =
(L1  L2)
L1 L2

Since L1 is regular, L1 is regular.

Since L2 is regular, L2 is regular.
 L1  L2 is regular.
 L1  L2 = (L1  L2) is regular.

17
 Closure of Regular Languages
Under Difference
L1 - L2 =

18
 Closure of Regular Languages
Under Difference
L1 - L2 = L1  L2

Since L2 is regular, L2 is regular.

Since both L1 and L2 are regular,

L1 - L2 = L1  L2 is regular.

19
 Letter Substitution
● Let 1 and 2 be alphabets.

● Let sub be any function from 1 to 2*.

Example:

Let: 1 = {a, b},

2 = {0, 1},

sub(a) = 0, and
sub(b) = 11.

20
 Letter Substitution
letsub is a letter substitution function iff:

letsub(L1) = {w  2* : y  L1 and

w = y except that:
every character c of y
is replaced by sub(c)}.

Example:

sub(a) = 0, and
sub(b) = 11.

Then letsub({anbn, n  0}) =

21
 Divide-and-Conquer
Let L = {w  {a, b}* : w contains an even number of a’s
and an odd number of b’s and all a’s come in runs of
three}.

L = L1  L2, where:
• L1 = {w  {a, b}* : w contains an even number of a’s
and an odd number of b’s}, and

• L2 = {w  {a, b}* : all a’s come in runs of three}

22
L1 is Regular

23
 L2 is Regular

Since both L1 and L2 are regular, L = L1  L2 is regular.

24
Don’t Try to Use Closure Backwards
One Closure Theorem:

If L1 and L2 are regular, then so is

L = L1  L2

But if L is regular, what can we say about L1 and L2?

L = L1  L2

25
Don’t Try to Use Closure Backwards
One Closure Theorem:

If L1 and L2 are regular, then so is

L = L1  L2

But if L is regular, what can we say about L1 and L2?

L = L1  L2

ab = ab  (a  b)* (they are regular)

26
Don’t Try to Use Closure Backwards
One Closure Theorem:

If L1 and L2 are regular, then so is

L = L1  L2

But if L is regular, what can we say about L1 and L2?

L = L1  L2

ab = ab  (a  b)* (they are regular)

ab = ab  {anbn, n  0} (they may not be regular)

27
Don’t Try to Use Closure Backwards
Another Closure Theorem:

If L1 and L2 are regular, then so is

L = L1 L2

But if L2 is not regular, what can we say about L?

L = L1  L2

28
Don’t Try to Use Closure Backwards
Another Closure Theorem:

If L1 and L2 are regular, then so is

L = L1 L2

But if L2 is not regular, what can we say about L?

L = L1 L2
{abanbn : n  0} = {ab} {anbn : n  0}

29
 Don’t Try to Use Closure Backwards
Another Closure Theorem:

If L1 and L2 are regular, then so is

L = L1 L2

But if L2 is not regular, what can we say about L?

L = L1 L2

Not
{abanbn : n  0} = {ab} {anbn : n  0}
Regular
{aaa*} = {a*} {ap: p is prime}
Regular 30
Showing that a Language is Not Regular

Every regular language can be accepted by some FSM.

It can only use a finite amount of memory to record

essential properties.

Example:
{anbn, n  0} is not regular

31
Showing that a Language is Not Regular
The only way to generate/accept an infinite language with
a finite description is to use:
• Kleene star (in regular expressions), or
• cycles (in automata).

This forces some kind of simple repetitive cycle within the

strings.

Example:
ab*a generates aba, abba, abbba, abbbba, etc.

Example:
{an : n  1 is a prime number} is not regular.

32
 How Long a String Can Be Accepted?

What is the longest string that a 5-state FSM can accept?

33
 How Long a String Can Be Accepted?

What is the longest string that a 5-state FSM can accept?

34
 How Long a String Can Be Accepted?

What is the longest string that a 5-state FSM can accept?

35
 Exploiting the Repetitive Property

If an FSM with n states accepts any string of length  n,

how many strings does it accept?
L = bab*ab
b a b a b
x y z We can We can also pump in
pump y out. as many y as we like..
xy*z must be in L.
So L includes: baab, babab, babbab, babbbbbbbbbbab
36
 Theorem – Long Strings
Theorem: Let M = (K, , , s, A) be any DFSM. If M
accepts any string of length |K| or greater, then that string
will force M to visit some state more than once (thus
traversing at least one loop).

Proof: M must start in one of its states. Each time it reads

an input character, it visits some state. So, in processing
a string of length n, M creates a total of n + 1 state visits.
If n+1 > |K|, then, by the pigeonhole principle, some
state must get more than one visit. So, if n  |K|, then M
must visit at least one state more than once.

37
The Pumping Theorem for Regular Languages

If L is regular, then every long string in L is pumpable.

So, k  1

( strings w  L, where |w|  k

( x, y, z (w = xyz,
|xy|  k,
y  , and
q  0 (xyqz is in L)))).

38
Example: {anbn: n  0} is not Regular
k is the number from the Pumping Theorem.
Choose w to be ak/2bk/2 (“long enough”).
1 2
aaaaa…aaaaabbbb …bbbbbb
x y z

Case III
We show that there is no x, y, z with the required properties:
|xy|  k,
y  ,
 q  0 (xyqz is in L).
Three cases to consider:
● Case I: y falls in region 1:
● Case II: y falls across regions 1 and 2:
● Case III: y falls in region 2: 39
Example: {anbn: n  0} is not Regular
k is the number from the Pumping Theorem.
Choose w to be ak/2bk/2 (“long enough”).

1 2
aaaaa…aaaaabbbb …bbbbbb
x y z

While this works, there is an easier way:

40
Example: {anbn: n  0} is not Regular
Second try:
Choose w to be akbk (We get to choose any w).

1 2
aaaaa…aaaaabbbb …bbbbbb
x y z

We show that there is no x, y, z with the required properties:

|xy|  k,
y  ,
 q  0 (xyqz is in L).
Since |xy|  k, y must be in region 1. So y = ap for some p  1.
Let q = 2, producing:
ak+pbk
which  L, since it has more a’s than b’s. 41
 A Complete Proof
We prove that L = {anbn: n  0} is not regular
If L were regular, then there would exist some k such that any string w where
|w|  k must satisfy the conditions of the theorem. Let w = ak/2bk/2.
Since |w|  k, w must satisfy the conditions of the pumping theorem. So,
for some x, y, and z, w = xyz, |xy|  k, y  , and q  0, xyqz is in L. We
show that no such x, y, and z exist. There are 3 cases for where y could
occur: We divide w into two regions:
aaaaa…..aaaaaa | bbbbb…..bbbbbb
1 | 2
So y can fall in:
● (1): y = ap for some p. Since y  , p must be greater than 0. Let q = 2.
The resulting string is ak+pbk. But this string is not in L, since it has
more a’s than b’s.
● (2): y = bp for some p. Since y  , p must be greater than 0. Let q = 2.
The resulting string is akbk+p. But this string is not in L, since it has
more b’s than a’s.
● (1, 2): y = apbr for some non-zero p and r. Let q = 2. The resulting
string will have interleaved a’s and b’s, and so is not in L.
There exists one long string in L for which no x, y, z exist. So L is not regular.
 Using the Pumping Theorem
If L is regular, then every “long” string in L is pumpable.

To show that L is not regular, we find one that isn’t.

To use the Pumping Theorem to show that a language L is

not regular, we must:

1. Choose a string w in L, where |w|  k. Since we do not

know what k is, we must state w in terms of k.
2. Divide the possibilities for y into a set of equivalence
classes that can be considered together.
3. For each such class of possible y values where |xy|  k
and y  :
Choose a value for q such that xyqz is not in L.
43
Bal = {w  {), (}* :the parens are balanced}
A w in L can be very complicated, but we only pick one.

If we pick (k)k, the proof is similar to the previous example.

44
 PalEven = {wwR : w  {a, b}*}
Again, a w in L can be very complicated, but we only pick one.

If we pick akbkbkak, the proof is similar to the previous examples.

45
 {anbm: n > m}
Sometimes we need to pump out.

Let w = ak+1bk, can we prove by pumping in?

Is w = a2kbk a good choice? 46

 L = {an: n is prime}
Let w = aj, where j is the smallest prime number > k+1.

y = ap for some p.

q  0 (a|x| + |z| +q|y| must be in L). So |x| + |z| +q|y| must be prime.

But suppose that q = |x| + |z|. Then:

|x| + |z| +q|y| = |x| + |z| + (|x| + |z|)y

= (|x| + |z|)(1 + |y|),

which is non-prime if both factors are greater than 1:

47
 L = {an: n is prime}
Let w = aj, where j is the smallest prime number > k+1.

y = ap for some p.

q  0 (a|x| + |z| +q|y| must be in L). So |x| + |z| +q|y| must be prime.

But suppose that q = |x| + |z|. Then:

|x| + |z| +q|y| = |x| + |z| + (|x| + |z|)y

= (|x| + |z|)(1 + |y|),

which is non-prime if both factors are greater than 1:

(|x| + |z|) > 1 because |w| > k+1 and |y|  k.

(1 + |y|) > 1 because |y| > 0.
48
Using the Pumping Theorem Effectively
● Tochoose w:
● Choose a w that is in the part of L that makes it not
regular.
● Choose a w that is only barely in L.
● Choose a w with as homogeneous as possible an
initial region of length at least k.

● To choose q:
● Try letting q be either 0 or 2.
● If that doesn’t work, analyze L to see if there is some
other specific value that will work.

49
 Using the Closure Properties

The two most useful ones are closure under:

• Intersection

• Complement

50
 Using the Closure Properties

L = {w  {a, b}*: #a(w) = #b(w)}

If L were regular, then:

L = L  _______

would also be regular. But it isn’t.

51
 L = {aibj: i, j  0 and i  j}

Try to use the Pumping Theorem by letting w = ak+1bk:

52
 L = {aibj: i, j  0 and i  j}

Try to use the Pumping Theorem by letting w = akbk+k!.

Then y = ap for some nonzero p.
Let q = (k!/p) + 1 (i.e., pump in (k!/p) times).
Note that (k!/p) must be an integer because p < k.

The number of a’s in the new string is k + (k!/p)p = k + k!.

So the new string is ak+k!bk+k!, which has equal numbers

of a’s and b’s and so is not in L.

53
 L = {aibj: i, j  0 and i  j}

An easier way:

If L is regular then so is L. Is it?

54
 L = {aibj: i, j  0 and i  j}

An easier way:

If L is regular then so is L. Is it?

L = {anbn: n  0}  {out of order}

If L is regular, then so is L = L  a*b*

= ___________

55
L = {aibjck: i, j, k ≥ 0 and (if i = 1 then j = k)}

Every string in L of length at least 1 is pumpable.

But is L regular?

56
L = {aibjck: i, j, k ≥ 0 and (if i = 1 then j = k)}

Every string in L of length at least 1 is pumpable.

Rewrite the final condition as: (i  1) or (j = k)

57
 L = {aibjck: i, j, k ≥ 0 and (i  1 or j = k)}
Every string in L of length at least 1 is pumpable:
•If i = 0 then: if j  0, let y be b; otherwise, let y be c. Pump
in or out. Then i will still be 0 and thus not equal to 1, so
the resulting string is in L.

•If i = 1 then: let y be a. Pump in or out. Then i will no

longer equal 1, so the resulting string is in L.
•
•If i = 2 then: let y be aa. Pump in or out. Then i cannot
equal 1, so the resulting string is in L.

•If i > 2 then: let y = a. Pump out once or in any number of

times. Then i cannot equal 1, so the resulting string is in L.

58
 L = {aibjck: i, j, k ≥ 0 and (i  1 or j = k)}

But the closure theorems help. Suppose we guarantee

that i = 1. If L is regular, then so is:
L = L  ab*c*.
L = {abjck : j, k ≥ 0 and j = k}
Use Pumping to show that L is not regular:

59
 L = {aibjck: i, j, k ≥ 0 and (i  1 or j = k)}
An Alternative
If L is regular, then so is LR:
LR = {ckbjai : i, j, k ≥ 0 and (i  1 or j = k)}
Use Pumping to show that L is not regular:

60
 {abanbn : n  0}
Choose w = abakbk

k
abaaaaaaaaaabbbbbbbbbbb
x y z

What are the choices for (x, y):

(, a)
(, ab)
(, aba+)
(a, b)
(a, ba+)
(aba*, a+)
61
 A Different Approach to abanbn?

Can we argue as follows:

We already know that {anbn: n  0} is not regular. So

neither is L.

Can we defend this argument by appeal to the fact that

the regular languages are closed under concatenation:

L = ab || {anbn: n  0}
? regular not regular

62
There is a Closure Theorem that Helps

{abanbn : n  0}

63
Exploiting Problem-Specific Knowledge
L = {w  {0, 1, 2, 3, 4 ,5, 6, 7}*:
w is the octal representation of a
nonnegative integer that is divisible by 7}

64
Exploiting Problem-Specific Knowledge
Let  = {w, h, q, e, x, r, l }.
Let L = {w : w represents a song in 4/4 time}.

65
 Is English Regular?
Is English finite?

66
In the event that the Purchaser defaults in the payment of any installment of purchase price, taxes,
insurance, interest, or the annual charge described elsewhere herein, or shall default in the
performance of any other obligations set forth in this Contract, the Seller may: at his option: (a)
Declare immediately due and payable the entire unpaid balance of purchase price, with accrued
interest, taxes, and annual charge, and demand full payment thereof, and enforce conveyance of
the land by termination of the contract or according to the terms hereof, in which case the
Purchaser shall also be liable to the Seller for reasonable attorney's fees for services rendered by
any attorney on behalf of the Seller, or (b) sell said land and premises or any part thereof at public
auction, in such manner, at such time and place, upon such terms and conditions, and upon such
public notice as the Seller may deem best for the interest of all concerned, consisting of
advertisement in a newspaper of general circulation in the county or city in which the security
property is located at least once a week for Three (3) successive weeks or for such period as
applicable law may require and, in case of default of any purchaser, to re-sell with such
postponement of sale or resale and upon such public notice thereof as the Seller may determine,
and upon compliance by the Purchaser with the terms of sale, and upon judicial approval as may
be required by law, convey said land and premises in fee simple to and at the cost of the Purchaser,
who shall not be liable to see to the application of the purchase money; and from the proceeds of
the sale: First to pay all proper costs and charges, including but not limited to court costs,
advertising expenses, auctioneer's allowance, the expenses, if any required to correct any
irregularity in the title, premium for Seller's bond, auditor's fee, attorney's fee, and all other
expenses of sale occurred in and about the protection and execution of this contract, and all
moneys advanced for taxes, assessments, insurance, and with interest thereon as provided herein,
and all taxes due upon said land and premises at time of sale, and to retain as compensation a
commission of five percent (5%) on the amount of said sale or sales; SECOND, to pay the whole
amount then remaining unpaid of the principal of said contract, and interest thereon to date of
payment, whether the same shall be due or not, it being understood and agreed that upon such
sale before maturity of the contract the balance thereof shall be immediately due and payable;
THIRD, to pay liens of record against the security property according to their priority of lien and to
the extent that funds remaining in the hands of the Seller are available; and LAST, to pay the
remainder of said proceeds, if any, to the vendor, his heirs, personal representatives, successors 67 or
assigns upon the delivery and surrender to the vendee of possession of the land and premises, less
costs and excess of obtaining possession.
 Is English Regular?
● The rat ran.
● The rat that the cat saw ran.
● The rat that the cat that the dog chased
saw ran.

Let:
A = {cat, rat, dog, bird, bug, pony}
V = {ran, saw, chased, flew, sang, frolicked}.

Let L = English  {The A (that the A)* V* V}.

L = {The A (that the A)n Vn V, n  0}.

Let w = The cat (that the rat)k sawk ran.

68
 Poetry
The Pumping Lemma
Any regular language L has a magic number p
And any long-enough word in L has the following property:
Amongst its first p symbols is a segment you can find
Whose repetition or omission leaves x amongst its kind.
So if you find a language L which fails this acid test,
And some long word you pump becomes distinct from all the
rest,
By contradiction you have shown that language L is not
A regular guy, resiliant to the damage you have wrought.
But if, upon the other hand, x stays within its L,
Then either L is regular, or else you chose not well.
For w is xyz, and y cannot be null,
And y must come before p symbols have been read in full.
As mathematical postscript, an addendum to the wise:
The basic proof we outlined here does certainly generalize.
So there is a pumping lemma for all languages context-free,
Although we do not have the same for those that are r.e.
69
-- Martin Cohn