Formal Language and Automata Theory

Mathematical preliminaries and notations

Sets
Relations and Functions
Graphs and Trees
Proof Techniques
The Basic Concepts of Automata
Alphabet
 String
Languages
 Grammars
 Automata
3. Some Applications
1
 Informally, a set is a (finite or infinite) collection of distinct
elements; the order of elements does not matter.
 For example, the collection of four letters a, b, c and d is a set,
which is written as
S= { a, b, c, d }
 The objects comprising a set are called its “elements” or
“members”.
 Singleton: A set having only one element.
 Empty set: a set with no element at all, which is denoted by {}
2
or ∅.
 Sets notation
 To indicate that x is a member of the set S, we write x
∈ S.
 If every element of set A is also an element of set B,
we say that A is a subset of B, and write A⊆ B.
 If every element of set A is also an element of set B,
but B also has some elements not contained in A, we
say that A is a “proper subset” of B and write A ⊂ B.

3
 Representations of sets
There are two methods to represent a set.
1. Roster method (Tabular form): a set is represented by listing all its elements,
separating by commas and enclosing in curly bracket.

If
 V be the set of vowels, it can be written as: V = { a, e, i, o, u}

If A be
 the set of natural numbers less than 7.

then A={1, 2, 3, 4, 5, 6}, is in the Roster form.

Note : To write a set in Roster form elements are not to be repeated i.e. all elements

are taken as distinct. For example if A be the set of letters used in the word

mathematics, then A = {m, a, t, h, e, i, c, s}

4
 Representations of sets
2. Set-builder form
In this form elements of the set are not listed but these are represented by some common property.

(i). Let V be the set of vowels of English alphabet then V can be written in the set builder form as: V =

{x : x is a vowel of English alphabet}

(ii) Let A be the set of natural numbers less than 7.

then A = {x : x Î N and 1 £ x <7}

Note : Symbol ':' read as 'such that'

Example: Write the following in set -builder form:

(a) A={- 3 -,2,-1,0,1,2,3} (b) B = {3,6,9,12}

Solution : (a) A = {x : xÎ Z and -3 £ x £ 3}

(b) B = {x : x = 3n and xÎN , n £ 4 }

5
set operations
The set operations are operations done on sets.
Union
The “union” of two sets is the set that has objects that are elements of at least one of
the two given sets, and possibly both.
That is, the union of sets A and B, written A ∪ B, is a set that contains everything in
A, or in B, or in both.
A∪B={x: x ∈ A or x ∈ B }
Example: A = {1, 3, 9} B = {3, 5}
Therefore, A ∪ B={1,3,5,9}

6
Intersection
 The “intersection” of sets A and B, written A ∩ B,
is a set that contains exactly those elements that
are in both A and B.
 A ∩ B = {x : x ∈ A and x ∈ B}
 Example1:
 Given A={1, 3, 9}, B = {3, 5}, C = {a, b, c}
A ∩ B = {3} Example 2. Consider the sets A =
{♣,♠} and B = {♣, ♦}. Then,
 A∩C={} • A ∪ B = {♣, ♠, ♦};
7 • A ∪ A = A ∪ ∅ = {♣, ♠}.
Set Difference
 The “set difference” of set A and set B, written as A–B, is
the set that contains everything that is in A but not in B.

A −B = {x : x ∈ A and x ∉ B}

Given A = {1 , 3 , 9} , B = {3 , 5}

A − B = {1 , 9 } .

8
Complement
 The complement of a set, denoted A', is the set of all
elements in the given universal set U that are not in A. 
 In set- builder notation, A' = {x: x ∈ U : x ∉ A}.
Example:
Let U = {1, 2, 3, 4, 5, 6} and A = {1, 3, 5}.
Then A' = {2, 4, 6}.  
U' = ∅ The complement of the universe is the empty set. 
∅' = U The complement of an empty set is the universal set.

 Example 2. Set the universe to U = {♣, ♠, ♥, ♦} and consider the set

A = {♣, ♦}. Then,
 = {♥, ♠};
 ∅ = {♣, ♠, ♥, ♦};
9
Properties of set operations
 For any three finite sets A, B and C;
1. Commutative Laws: A U B = B U A
A∩ B=B∩A
2. Associative Laws: (A U B) U C = A U (B U C)
(ii) (A ∩ B) ∩ C = A ∩ (B ∩ C)
Thus,union and intersection are associative.
3. Idempotent Laws: A U A = A
A∩A=A
4. Absorption : (A ∪ B) ∩ A = A
(A ∩ B) ∪ A = A

10
Example 1 :
 Given, A = {1, 2, 3, 4, 5}, B = {3, 4, 5, 6} and C = {5, 6, 7, 8},
show that
(i) A U (B U C) = (A U B) U C. (ii) Verify (i) using Venn
diagram.
Solution :
 Now, B U C = {3, 4, 5, 6} U {5, 6, 7, 8} = {3, 4, 5, 6, 7, 8}

= A U (B U C)  =  {1, 2, 3, 4, 5} U { 3, 4, 5, 6, 7, 8}
=  {1, 2, 3, 4, 5, 6, 7, 8}  -----  (1)
 Now, (A U B) = {1, 2, 3, 4, 5} U {3, 4, 5, 6} = {1,2,3,4,5,6}
 (A U B) U C = {1,2,3,4,5,6} U {5,6,7,8}
=  {1, 2, 3, 4, 5, 6, 7, 8}   -----  (2)
= (1)  =  (2)

11
Using venn diagram

12
 Example 2 :
 Let A = {a,b,c,d}, B = {a,c,e} and C = {a,e}.
(i) Show that A n (B n C) = (A n B) n C.
(ii)(ii) Verify (i) using Venn diagram
Solution :
 Now, B n C = {a,c,e} n {a, e} = {a, e}
 A n (B n C)  =  {a, b, c ,d} U {a, e}

=  { a }  -----  (1)
 (A n B)  =  {a,b,c,d} n {a,c,e}

=  {a, c}
 (A n B) n C  =  {a, c} n {a,e}
=  {a}  -----  (2)
(1)  =  (2)

13
Example 3:
 Use the Venn diagram to answer the following
questions
 (i) List the elements of U, E', F', (E U F)' and (E n F)'

14
i. To find the elements of universal set U, we have to list out all the
elements that we find in the rectangular box.
U = {1, 2, 3, 4, 7, 9, 10, 11}
ii. To find the elements of the set E', we have to list out all the
elements other than the circle E.
E' = {3, 9, 10, 11 }
iii. To find the elements of the set F', we have to list out all the
elements other than the circle F.
F' = { 1, 2, 3, 4, 7, 10 }
iv. To find the elements of the set (E U F)', first we have to find the
elements of the set E U F
E U F =  {1, 2, 4, 7, 9, 11}
(E U F)' =  {3, 10}
v. To find the elements of the set (E n F)', first we have to find the
elements of the set E n F
E n F =  {4, 7}
(E n F)' =  {1, 2, 3, 9, 10, 11}
15
Properties of set operations

4. Distributive Laws: A U (B ∩ C) = (A U B) ∩ (A U C)
A ∩ (B U C) = (A ∩ B) U (A ∩ C)
 Thus, union and intersection are distributive over
intersection and union respectively.

5. De Morgan’s Laws: A – (B U C) = (A – B) ∩ (A – C)

A - (B ∩ C) = (A – B) U (A – C)
 De Morgan’s Laws can also we written as:

(A U B)’ = A' ∩ B’

(A ∩ B)’ = A' U B'

16
Additional Terminology

 Disjoint Sets: If A and B have no common element, that is, A ∩ B

= ∅, then the sets A and B are said to be disjoint.

 Cardinality: The “Cardinality” of a set A, written |A|, is the

number of elements in set A.

 Powerset: The “powerset” of a set A, written 2 A, is the set of all

subsets of A; i.e., a set containing ‘n’ elements has a powerset

containing 2n elements.

17
 Con…

 Cartesian Product: Let A and B be two sets; Then the set

of all ordered pairs (x, y) where x ∈ A and y∈ B is called

the “Cartesian Product” of the sets A and B and is

denoted by A x B , i.e.

A x B = {(x , y) : x ∈ A and y ∈ B}

18
Examples
1: Given A = {1, 2, 3} determine P(A) (powerset of A).
As the set A = {1, 2, 3} has 3 elements the powerset P(A) will have 2 3 = 8 elements.

P (A) = {∅,{1 }, {2 }, {3 }, {1 , 2 }, {1 , 3}, {2 , 3 }, {1 ,2 ,3 }}

2. Given A={1,2}, and B={a,b}
Cardinality of set A is |A|=2

A ∩B= ∅ A and B are disjoint sets.

The cartesian products of A and B.

AxB={(1,a),(1,b),(2,a),(2,b)}
3. Given A= {1,2}, B = {x, y, z} and C = {3, 4}, find A × B × C and
n (A × B × C ) .

19
Relations and Functions
Relation: A relation on sets S and T is a set of ordered pairs (s, t), where
(a) s∈ S (s is a member of S )

(b) t ∈T (t is a member of T )

(c) S and T need not be different

(d) The set of all first elements is the domain of the relation,

(e) The set of all second elements is the range of the relation.

20
21
Equivalence Relation
 A subset R of AxA is called an equivalence relation on
A if R satisfies the following conditions:
i (a, a ) ∈ R for all a ∈ A (R is reflexive)
ii If (a, b ) ∈ R, and (b, a ) ∈ R, then
(R is symmetric)
iii If (a, b ) ∈ R and (b, c) ∈ R, then
(a, c) ∈ R (R is transitive)
Example:
Given the relation R in A as
R = {(1,1), (2,2), (2,3), (3,2), (4,2), (4,4)}
(a) Is R (i) reflexive (ii) symmetric (iii) transitive?
No why ?
22
 Functions as relations
 A function f : A → B is a function from A to B i.e., a subset of A

× B , such that each a∈ A belongs to a unique ordered pair (a,

b) in f.
 Kinds of Functions
 One-to-One Function (Injection):
A function f: A →B is said to be one-to-one if different elements
in the domain A have distinct images in the range.

A function f is one-to-one if f (a ) = f(a′) implies a= a′.

23
Onto function (Surjection):
 A function f : A→B is said to be an onto function if each

element of B is the image of some element of A.

 f : A→B is onto if the image of f is the entire codomain.

 If f (A)=B. i.e., f maps A onto B.

24
One-to-one onto Function (Bijection):
A function that is both one-to-one and onto is called a “Bijection”.
Such a function maps each and every element of A to exactly one element of

B, with no elements left over.

25
 Examples
.

26
Boolean logic
Boolean logic is a system built with two values TRUE and FALSE.
“Boolean values” are represented by values 0 and 1.
There are many Boolean operations.
Negation: It means the NOT operation, represents by ¬. Example: ¬ 0 =1 and ¬1= 0.
Conjunction: It means the AND operation, represented by ∧.
Disjunction: It means the OR operation, represented by ∨

27
Boolean logic
Equality: The equality operation, written with the symbol ↔, is 1 if both its operands have the

same value.
Implication: This operation is designated by the symbol → and is 0 if its first operand is 1 and

its second operand is 0; otherwise → is 1.

Exclusive-OR operation: 1 if either but not both of its operands are 1. Exclusive-OR is


denoted by ⊕.

28
Fundamental Proof Techniques
 A proof is a sequence of steps that begins from some known facts
to reach the desired conclusions; Each step must be obviously
correct.
 Techniques
 Proof by contradiction
 To prove some fact p, we show ‘not p’ is false.
 Proof by Induction
 We show that some fact is true for every natural number n, using
two arguments.
 Base
 Step
 Pigeon hole principle
29
Principle of Mathematical Induction

 Base: Consider an initial value for which the statement is true.

It is to be shown that the statement is true for n= initial value.
 Step: Assume the statement is true for any value of n = k. Then
prove the statement is true for n = k+1.
 We actually break n = k+1 into two parts, one part is n = k (which
is already proved) and try to prove the other part.

30
Examples Mathematical induction
Let Pn be a proposition that depends on n Z+.

Then Pn is true for all +ve n provided that:

i. Pi is true

ii. If Pk is true, so is Pk+1, for some k Z+.

Three steps:

1. Base case: verify that P1 holds true

2. Inductive hypothesis: assume that Pk holds true, for some k Z+

3. Inductive step: show that Pk+1 holds

Ex. Show that 1+2+…+n = n(n+1)/2, for all n Z+.

31
 Solution:
Let Pn: 1+2+…+n = n(n+1)/2
Step1: for n = 1, P1 holds true
Step2: for some k Z+, assume Pk is true
i.e. Pk: 1+2+…+k = k(k+1)/2
Pk
Step3: WTS Pk+1 is true
Pk+1 : 1+2+…+k+(k+1) = (k+1)(k+2)/2
: Pk + (k+1) = (k+1)(k+2)/2
: k(k+1)/2 + (k+1) = (k+1)(k+2)/2
: [k(k+1) + 2(k+1)]/2 = (k+1)(k+2)/2
: (k+1)(k+2)/2 = (k+1)(k+2)/2
Therefore, Pn holds true for all nZ+
Ex.32Show that Pn = (i=1,n), i2 = (n)(n+1)(2n+1)/6 for all n
(b) Pigeon-hole Principle

 If A and B are finite sets and |A| > |B|, then there is no one-to-
one function from A to B. i.e., If an attempt is made to pair off
the elements of A (the “pigeons”) with elements of B (the
“pigeonholes”), sooner or later we will have to put more than
one pigeon in a pigeonhole.
By induction, the pigeonhole principle can be proved.

33
Graphs and trees
 A graph, is a collection of points connected by lines.
 The points are called nodes or vertices, and the lines are
called edges

34
 Graphs and trees: cont’d
 A directed graph has arrows instead of lines, as shown in the
following figure.
 The number of arrows pointing from a particular node is the out
degree of that node, and the number of arrows pointing to a
particular node is the in degree.

35
 Graphs and trees: cont’d
 The degree of a vertex v in a graph (directed or undirected) is the
number of edges with v as an end vertex.

 Note: that a self loop is counted twice when calculating the

degree of a vertex.
Ex. In the previous graph, deg(v1) = ? deg(v2) = ?

 A path may be directed (if all the edges in the path have the same
direction.)

36
Trees
 A graph (directed or undirected) is called a tree if it is connected
and has no circuits.
Q. Are the previous two graphs trees?
Properties of trees:
 In a tree there is one and only one path between every pair of
vertices (nodes)
 A tree with n vertices has n-1 edges
 A leaf in a tree can be defined as a vertex of degree one
37 Vertices other than leaves are called internal vertices
Graphs and trees: cont’d

38
Class work

39
 Automata theory
.

 Automaton theory: Study of abstract computing devices, or

machines
 Automata theory deals with the definitions and properties of
mathematical models of computation.
 Automata : is a self-operating machine, or a machine or control
mechanism designed to follow automatically a predetermined
sequence of operations, or respond to predetermined instructions.

40
 Automaton
.

 Automaton = an abstract computing device

 Note: A device need not even be a physical hardware!
 Automata means something that works automatically.
 Automata are a model that used in text processing, compilers,
and hardware design called the finite automaton
 Another model, called the context-free grammar, is used in
programming languages and artificial intelligence.

41
 Con…
.

 A fundamental question in computer science are:

 Find out what different models of machines can do and
cannot do?

 The theory of computation

 Computability vs. Complexity

 Computability theory: what kinds of things can you

solve with a computer and what kinds of things can’t
you solve with a computer?
42
 Con…
.

 Complexity theory: focuses on how hard is problem to solve using

a computer?
 We’ll define different ways of measuring hardness: time and space
(memory)
 The time complexity of an execution is the number of steps until the
machine halts.
 The space complexity is the number of tape cells used by the computation.
 Example is factorial number

 A famous unsolved problem is the P vs. NP problem, which will

be a theme of complexity.
43
 Con…
.

A pioneer of automata theory

ALAN TURING (1912-1954)

Father of Modern Computer Science

English mathematician

Studied abstract machines called Turing machines

even before computers existed

44
 Con…
.

45
 Con…
 Halting— means that the program on certain input will accept
it and halt or reject it and halt.
 Basically halting means terminating.

 Halting Problem: There does not exist a Turing machine

that always halts.
 Undecidable: there is no Turing machine that decides it.
 Intractable: Certain computational problems are solvable in principle, but
the solutions require so much time or space that they can’t be used in
practice.
46
 Con…
.

47
 THE CENTRAL CONCEPTS OF AUTOMATA THEORY
.

In this section, we define the basic notions of the subject:

symbols, strings, alphabets and (formal) languages
The mathematical study of the “Theory of Computation” begins by
understanding the Mathematics of strings of symbols.

* A symbol is one of the finite sequences of ASCII characters:

* One of the digits 0–9;
* One of the upper case letters A–Z;
* One of the lower case letters a–z;
* A (, followed by any finite sequence of printable ASCII characters in

48 which ( and) are properly nested, followed by a)

 Alphabet & String
.

Alphabet: A finite non-empty set of symbols (called letters), is called an

alphabet. It is denoted by Σ (Greek letter sigma).
Example
Σ = {a,b}, Σ = {i,j,k}
String: A “string” over an alphabet is a finite sequence of symbols from that
alphabet, which is usually written next to one another and not separated by
commas.
(i) If Σa = {0 ,1}then 001001 is a string over Σa
(ii) If Σb = {a ,b , , , z) then axyrpqstcd is a string over Σb.

49
 String operations
.
Length of String

50
 String
.

 Empty String: The string of zero length is called the empty

string. This is denoted by ∈.
 The empty string plays the role of 0 in a number system.
 Reverse String: If w=w1w2...wn where each wi ∈ Σ, the
reverse of w is wnwn-1... w1.

51
 Substring:
.

z is a substring of w if z appears consecutively within w.

Example deck is a substring of abcdeckabcjkl.

Concatenation: Assume a string x of length m and string y of

length n, the concatenation of x and y is written xy, which is the

string obtained by appending y to the end of x, as in x1x2 ...xmy1 y2.....

yn .

To concatenate a string with itself many times we use the

“superscript” notation:
52
 Substring:
.

Suffix: If w=xv for some x, then v is a suffix of w.

Prefix: If w=vy for some y, then v is a prefix of w

Lexicographic ordering: The Lexicographic ordering of strings

is the same as the dictionary ordering, except that shorter

strings precede longer strings. The lexicographic ordering of all

strings over the alphabet {0, 1} is (∈, 0,1, 00, 01, 10, 11,

000, ...).

53
 Summary
.

54
 Language
.

Language: Any set of strings over an alphabet Σ is called a language.

A language L is regular if it is the language accepted by some DFA
The set of all strings, including the empty string over an alphabet Σ is

denoted as Σ*.
Infinite languages L are denoted as: L={w∈ Σ*: has property P}
Examples:
a) L1 = {w ∈{0,1}* : has an equal number of 0's and 1's}
b) L2 = {w∈ Σ* : w= wR } where wR is the reverse string of w.

55
 Language
.

Concatenation of Languages: If L1 and L2 are languages over Σ,

their concatenation is L = L1•L2, or simply L =L1L2, where

L = {w ∈ Σ* :w = x •y for some x∈ L1 and y∈L2}.

Example: Given Σ = {0, 1}

L1 = {w ∈ Σ* : w has an even number of 0's}

L 2 = { w: w starts with a 0 and the rest of the symbols are 1's}

then L1 L2 = {w : w has an odd number of 0's}

56
 Language
.

Kleene Star: Another language operation is the “Kleene Star” of a

language L, which is denoted by L*.

L* is the set of all strings obtained by concatenating zero or more

strings from L.

L*= w ∈ Σ*: w=w1....w k for some k ≥0 and some w1,w2,...,wk ∈ L

Example: If L = {01, 1, 100} then 110001110011∈ L*, since 110001110011 =

1• 100• 01• 1• 100 • 1• 1, each of these strings is in L.

57
 Example
.

Given Σ = {a, b} obtain Σ*.

(a) Give an example of a finite language in Σ.

(b) Given L = {anbn : n ≥0 } , check if the strings aabb, aaaabbbb, abb are in the language L.

Solution: Σ = {a, b} Therefore we have

Σ* = {λ , a, b, aa ,ab, ba, bb, aaa }

(a) {a, aa, aab} is an example of a finite language in Σ.

(i) aa bb → a string in L. (n = 2)

(ii) aaaa bbbb → a string in L. (n = 4)

(iii) abb → not a string in L (since there is no n satisfying this).

58
 Summary
.

59
 Grammars
.

 Grammar is a mechanism to describe the languages.

 A grammar (G) is defined as a quadruple
G = (V, T, S, P) where
V = Finite set of objects called VARIABLES
T = Finite set of objects called TERMINAL SYMBOLS
S ∈V = Start variables
P = Finite set of Productions.
A production rule P is of the form x→y
 Given a string w, of the form w = uxv, we can use the
production rule x → y and obtain a new string z = uyv.

60
 Grammars
.

The set of all strings obtained by using production rules is the Language generated by

the Grammar.
If the grammar G = (V, T, S, P) then

L(G ) = {w∈T* :S w}

If W ∈ L(G) , then the sequence

S ⇒w1 ⇒w2 ⇒ w3 . . .⇒ wn ⇒ w is a “derivation” of the sentence w.

The string S, w1, w2, wn which contain variables as well as terminals, are called

“SENTENTIAL FORMS” of the derivation.

61
.
Grammars
Example: Given a Grammar
G = ({S } , {a ,b}, S , P) with P defined as
S →aSb,
S→λ

i. Obtain a sentence in the language generated by G and the sentential form

ii. Obtain the language L(G).

62
.
Grammars
Applying the production rule S → aSb , we get
ai S bi ⇒ ai+1 S bi+1
This is true for all i.
In order to get a sentence we apply S → λ.
Therefore we get S an S bn ⇒an bn
There fore L(G) = {anbn ;n ≥ 0 }.

63
.
Grammars

64
 Summary
.

65
 Practical Examples
.

 On/Off switch
action

state

 Modeling recognition of the word “then”

Start state Transition Intermediate Final state

state

66
 Structural expressions
.

 Grammars
 Regular expressions
E.g. Unix style to capture city names such as “Palo Alto CA”:
 [A-Z][a-z]*([ ][A-Z][a-z]*)*[ ][A-Z][A-Z]

Start with a letter

A string of other
letters (possibly empty) Should end w/ 2-letter state code

Other space delimited words

(part of city name)

67
 Some Applications
.

 Software for designing and checking the behavior of digital

circuits
 Lexical analyzer of a typical compiler
 Software for scanning large bodies of text (e.g., web pages),
for pattern finding
 Software for verifying systems of all types that have a finite
number of states (e.g., stock market transaction,
communication/network protocol)

68
Exercise
Given
   L1={ab,bc,ca}, L2={aa,ac,cb} determine
a. L1L2
b. L1
c. L1- L2
d. L1L2

69
.

Questions ?

70