https://www.tutorialspoint.

com/automata_theory/regu
lar_expressions.htm
Automata Theory Tutorial
PDF Version Quick Guide Resources Job Search Discussion
Automata Theory is a branch of computer science that deals with
designing abstract self-propelled computing devices that follow a
predetermined sequence of operations automatically. An automaton with a
finite number of states is called a Finite Automaton. This is a brief and
concise (краткий) tutorial that introduces the fundamental concepts of
Finite Automata, Regular Languages, and Pushdown Automata before
moving onto Turing machines and Decidability.

Audience
This tutorial has been prepared for students pursuing a degree in any
information technology or computer science related field. It attempts to help
students grasp the essential concepts involved in automata theory.

Prerequisites
This tutorial has a good balance between theory and mathematical rigor
(строгость). The readers are expected to have a basic understanding of
discrete mathematical structures.

Automata − What is it?

The term "Automata" is derived from the Greek word "αὐτόματα" which
means "self-acting". An automaton (Automata in plural) is an abstract self-
propelled computing device which follows a predetermined sequence of
operations automatically.

An automaton with a finite number of states is called a Finite

Automaton (FA) or Finite State Machine (FSM).

Formal definition of a Finite Automaton

An automaton can be represented by a 5-tuple (Q, ∑, δ, q0, F), where −

 Q is a finite set of states.

 ∑ is a finite set of symbols, called the alphabet of the automaton.
 δ is the transition function.
 q0 is the initial state from where any input is processed (q0 ∈ Q).
 F is a set of final state/states of Q (F ⊆ Q).

Related Terminologies
Alphabet
 Definition − An alphabet is any finite set of symbols.
  Example − ∑ = {a, b, c, d} is an alphabet set where ‘a’, ‘b’, ‘c’, and ‘d’ are alphabets.

String
 Definition − A string is a finite sequence of symbols taken from ∑.

 Example − ‘cabcad’ is a valid string on the alphabet set ∑ = {a, b, c,d}

Length of a String
 Definition − It is the number of symbols present in a string. (Denoted by |S|).

 Examples −

o If S=‘cabcad’, |S|= 6

o If |S|= 0, it is called an empty string (Denoted by λ or ε)

Kleene Star
 Definition − The set ∑* is the infinite set of all possible strings of all possible

lengths over ∑ including λ.

 Representation − ∑* = ∑0 ∪ ∑1 ∪ ∑2 ∪…….

 Example − If ∑ = {a, b}, ∑*= {λ, a, b, aa, ab, ba, bb,………..}

Kleene Closure/Plus
 Definition − The set ∑+ is the infinite set of all possible strings of all possible lengths over ∑ excluding λ.

 Representation − ∑+ = ∑0 ∪ ∑1 ∪ ∑2 ∪…….

∑+ = ∑* − { λ }

 Example − If ∑ = { a, b } , ∑+ ={ a, b, aa, ab, ba, bb,………..}

Language
 Definition − A language is a subset of ∑* for some alphabet ∑. It can be finite or infinite.

 Example − If the language takes all possible strings of length 2 over ∑ = {a, b}, then L = { ab, bb, ba, bb}

Deterministic Finite Automaton

Finite Automaton can be classified into two types −

 Deterministic Finite Automaton (DFA)

 Non-deterministic Finite Automaton (NDFA / NFA)

Deterministic Finite Automaton (DFA)

In DFA, for each input symbol, one can determine the state to which the
machine will move. Hence, it is called Deterministic Automaton. As it has
a finite number of states, the machine is called Deterministic Finite
Machine or Deterministic Finite Automaton.

Formal Definition of a DFA

A DFA can be represented by a 5-tuple (Q, ∑, δ, q0, F) where −

 Q is a finite set of states.

 ∑ is a finite set of symbols called the alphabet.

 δ is the transition function where δ: Q × ∑ → Q

 q0 is the initial state from where any input is processed (q0 ∈ Q).

 F is a set of final state/states of Q (F ⊆ Q).

Graphical Representation of a DFA

A DFA is represented by digraphs called state diagram.

 The vertices (вершины) represent the states.

 The arcs (дуги) labeled with an input alphabet show the transitions.
 The initial state is denoted by an empty single incoming arc.
 The final state is indicated by double circles.

Example

Let a deterministic finite automaton be →

 Q = {a, b, c}, is a finite set of states;

 ∑ = {0, 1}, finite set of symbols;
 q0={a}, initial state
 F={c}, F is a set of final state, and
 Transition function δ as shown by the following table –

Present State Next State for Input 0 Next State for Input 1

A a b

B c a

C b c

Its graphical representation would be as follows −

 Non-deterministic Finite Automaton (NDFA,)
In NDFA, for a particular input symbol, the machine can move to any
combination of the states in the machine. In other words, the exact state to
which the machine moves cannot be determined. Hence, it is called Non-
deterministic Automaton. As it has finite number of states, the machine
is called Non-deterministic Finite Machine or Non-deterministic Finite
Automaton.

Formal Definition of an NDFA

An NDFA can be represented by a 5-tuple (Q, ∑, δ, q0, F) where −

 Q is a finite set of states.

 ∑ is a finite set of symbols called the alphabets.

 δ is the transition function where δ: Q × {∑ ∪ ε} → 2Q (Here the power set of Q (2 Q)

has been taken because in case of NDFA, from a state, transition can occur to any
combination of Q states)

 q0 is the initial state from where any input is processed (q0 ∈ Q).

 F is a set of final state/states of Q (F ⊆ Q).

Graphical Representation of an NDFA − (same as DFA)

An NDFA is represented by digraphs called state diagram.

 The vertices represent the states.

 The arcs labeled with an input alphabet show the transitions.
 The initial state is denoted by an empty single incoming arc.
 The final state is indicated by double circles.

Example

Let a non-deterministic finite automaton be →

 Q = {a, b, c}
 ∑ = {0, 1}
 q0 = {a}
 F={c}
  The transition function δ as shown below −

Present State Next State for Input 0 Next State for Input 1

a a, b b

B c a, c

c b, c c

Its graphical representation would be as follows −

DFA vs NDFA
The following table lists the differences between DFA and NDFA.

DFA NDFA

The transition from a state is to a The transition from a state can be

single particular next state for each to multiple next states for each
input symbol. Hence it is input symbol. Hence it is
called deterministic. called non-deterministic.

Empty string transitions are not seen NDFA permits empty string
in DFA. transitions.

Backtracking is allowed in DFA In NDFA, backtracking is not

always possible.

Requires more space. Requires less space.

A string is accepted by a DFA, if it A string is accepted by a NDFA, if

transits to a final state. at least one of all possible
transitions ends in a final state.

Acceptors, Classifiers, and Transducers

Acceptor (Recognizer)
An automaton that computes a Boolean function is called an acceptor. All
the states of an acceptor is either accepting or rejecting the inputs given to
it.

Classifier
A classifier has more than two final states and it gives a single output
when it terminates.

Transducer
An automaton that produces outputs based on current input and/or previous
state is called a transducer. Transducers can be of two types −

 Mealy Machine − The output depends both on the current state and the current input.

 Moore Machine − The output depends only on the current state.

Acceptability by DFA and NDFA

A string is accepted by a DFA/NDFA iff the DFA/NDFA starting at the initial
state ends in an accepting state (any of the final states) after reading the
string wholly.

A string S is accepted by a DFA/NDFA (Q, ∑, δ, q0, F), iff

δ*(q0, S) ∈ F

The language L accepted by DFA/NDFA is

{S | S ∈ ∑* and δ*(q0, S) ∈ F}

A string S′ is not accepted by a DFA/NDFA (Q, ∑, δ, q0, F), iff

δ*(q0, S′) ∉ F

The language L′ not accepted by DFA/NDFA (Complement of accepted

language L) is

{S | S ∈ ∑* and δ*(q0, S) ∉ F}

Example
Let us consider the DFA shown in Figure 1.3. From the DFA, the acceptable
strings can be derived.
Strings accepted by the above DFA − {0, 00, 11, 010, 101, ...........}

Strings not accepted by the above DFA − {1, 011, 111, ........}

Problem Statement
Let X = (Qx, ∑, δx, q0, Fx) be an NDFA which accepts the language L(X). We
have to design an equivalent DFA Y = (Qy, ∑, δy, q0, Fy) such that L(Y) =
L(X). The following procedure converts the NDFA to its equivalent DFA −

Algorithm
Input: NDFA
Output
equivalent DFA
:
Step 1 Create state table from given NDFA.
Create a blank state table under possible input alphabets for
Step 2
equivalent DFA.
Step 3 Mark start state of DFA by q0 (same as NDFA).
Step 4 Find out the combination of States {Q0, Q1,... , Qn} for each
possible input alphabet.
Step 5 Each time we generate a new DFA state under the input alphabet
columns, we have to apply step 4 again, otherwise go to step 6.
Step 6 States which contain any of final states of NDFA are final states of
equivalent DFA.
Example
The NDFA table is as follows −

q δ(q,0) δ(q,1)

a {a,b,c,d,e} {d,e}

b {c} {e}

c ∅ {b}

d {e} ∅
 e ∅ ∅

Let us consider the NDFA shown in the figure below.

Using above algorithm, we find its equivalent DFA. The state table of the DFA is shown in
below.

q δ(q,0) δ(q,1)

a {a,b,c,d,e} {d,e}

{a,b,c,d,e} {a,b,c,d,e} {b,d,e}

{d,e} e ∅

{b,d,e} {c,e} E

e ∅ ∅

d e ∅

{c,e} ∅ B

b c E

c ∅ B

The state diagram of the DFA is as follows −

 Previous Page

Next Page

DFA Minimization

DFA Minimization using Myphill-Nerode

Theorem
Algorithm
Input: DFA
Output
Minimized DFA
:
Step 1 Draw a table for all pairs of states (Qi, Qj) not necessarily connected directly [All are unmarked initially]
Step 2 Consider every state pair (Qi, Qj) in the DFA where Qi ∈ F and Qj ∉ F or vice versa and mark them. [Here F is the
set of final states].
Step 3 Repeat this step until we cannot mark anymore states −

If there is an unmarked pair (Q i, Qj), mark it if the pair {δ(Qi, A), δ (Qi, A)} is marked for some input alphabet.

Step 4 Combine all the unmarked pair (Q i, Qj) and make them a single state in the reduced DFA.

Example
Let us use above algorithm to minimize the DFA shown below.
Step 1 − We draw a table for all pair of states.

a b c d e f

Step 2 − We mark the state pairs −

a b c d E f

c ✔ ✔

d ✔ ✔

e ✔ ✔

f ✔ ✔ ✔

Step 3 − We will try to mark the state pairs, with green colored check mark,
transitively. If we input 1 to state ‘a’ and ‘f’, it will go to state ‘c’ and ‘f’ respectively. (c, f) is
already marked, hence we will mark pair (a, f). Now, we input 1 to state ‘b’ and ‘f’; it will go to
state ‘d’ and ‘f’ respectively. (d, f) is already marked, hence we will mark pair (b, f).

a b c d E f
 a

c ✔ ✔

d ✔ ✔

e ✔ ✔

f ✔ ✔ ✔ ✔ ✔

After step 3, we have got state combinations {a, b} {c, d} {c, e} {d, e}
that are unmarked.

We can recombine {c, d} {c, e} {d, e} into {c, d, e}

Hence we got two combined states as − {a, b} and {c, d, e}

So the final minimized DFA will contain three states {f}, {a, b} and {c, d, e}

DFA Minimization using Equivalence Theorem

If X and Y are two states in a DFA, we can combine these two states into {X,
Y} if they are not distinguishable. Two states are distinguishable, if there is
at least one string S, such that one of δ (X, S) and δ (Y, S) is accepting and
another is not accepting. Hence, a DFA is minimal if and only if all the states
are distinguishable.

Algorithm
Step 1 All the states Q are divided in two partitions − final states and non-final states and are denoted by P0. All the
states in a partition are 0th equivalent. Take a counter k and initialize it with 0.
Step 2 Increment k by 1. For each partition in Pk, divide the states in Pk into two partitions if they are k-distinguishable.
Two states within this partition X and Y are k-distinguishable if there is an input S such that δ(X, S) and δ(Y,
S) are (k-1)-distinguishable.
Step 3 If Pk ≠ Pk-1, repeat Step 2, otherwise go to Step 4.
Step 4 Combine kth equivalent sets and make them the new states of the reduced DFA.

Example
Let us consider the following DFA −
 q δ(q,0) δ(q,1)

a b c

b a d

c e f

d e f

e e f

f f f

Let us apply above algorithm to the above DFA −

 P0 = {(c,d,e), (a,b,f)}

 P1 = {(c,d,e), (a,b),(f)}
 P2 = {(c,d,e), (a,b),(f)}

Hence, P1 = P2.

There are three states in the reduced DFA. The reduced DFA is as follows −

The State table of DFA is as follows −

Q δ(q,0) δ(q,1)

(a, b) (a, b) (c,d,e)

(c,d,e) (c,d,e) (f)

(f) (f) (f)

Its graphical representation would be as follows −

Mealy machine
In the theory of computation, a Mealy machine is a finite-state machine whose output values are determined both by its current state and
the current inputs. (This is in contrast to a Moore machine, whose output values are determined solely by its current state.) A Mealy machine
is adeterministic finite-state transducer: for each state and input, at most one transition is possible.

History[edit]
The Mealy machine is named after George H. Mealy, who presented the concept in a 1955 paper, “A Method for Synthesizing Sequential
Circuits”.[1]

Formal definition[edit]
A Mealy machine is a 6-tuple consisting of the following:

 a finite set of states

 a start state (also called initial state) which is an element of

 a finite set called the input alphabet

 a finite set called the output alphabet

 a transition function mapping pairs of a state and an input symbol to the corresponding next state.

 an output function mapping pairs of a state and an input symbol to the corresponding output symbol.

In some formulations, the transition and output functions are coalesced into a single function .

Comparison of Mealy machines and Moore

machines[edit]
1. Mealy machines tend to have fewer states:
 Different outputs on arcs (n2) rather than states (n).
2. Moore machine are safer to use:
  Outputs change at clock edge (always one cycle later).
 In Mealy machines, input change can cause output change as soon as logic is done—a big problem when two
machines are interconnected – asynchronous feedback may occur if one isn't careful.
3. Mealy machine react faster to inputs:
 React in same cycle—don't need to wait for clock.
 In Moore machines, more logic may be necessary to decode state into outputs—more gate delays after clock edge.

Diagram[edit]
The state diagram for a Mealy machine associates an output value with each transition edge (in contrast to the state diagram for a Moore
machine, which associates an output value with each state).

When the input and output alphabet are both Σ, one can also associate to a Mealy Automata an Helix directed graph.[2] (S × Σ, (x, i) →
(T(x, i), G(x, i)))
This graph has as vertices the couples of state and letters, every nodes are of out-degree one, and the successor of (x, i) is the next

state of the automata and the letter that the automata output when it is instate x and it reads letter i. This graph is a union of disjoint cycles if
the automaton is bireversible.

Examples[edit]
Simple[edit]

State diagram for a simple Mealy machine with one input and one output.

A simple Mealy machine has one input and one output. Each transition edge is labeled with the value of the input (shown in red) and the
value of output (shown in blue).

The machine starts in state Si. (In this example, the output is the exclusive-or of the two most-recent input values;

thus, the machine implements an edge detector, outputting a one every time the input flips.

and a zero otherwise.)

Complex
More complex Mealy machines can have multiple inputs as well as multiple outputs.

Applications[edit]
Mealy machines provide a rudimentary mathematical model for cipher machines. Considering the input and output alphabet the Latin
alphabet, for example, then a Mealy machine can be designed that given a string of letters (a sequence of inputs) can process it into a
ciphered string (a sequence of outputs). However, although one could use a Mealy model to describe the Enigma, the state diagram would
be too complex to provide feasible means of designing complex ciphering machines.
Moore/Mealy machines, are DFAs that have also output at any tick of the clock. Modern CPUs, computers, cell phones, digital clocks and
basic electronic devices/machines have some kind of finite state machine to control it.

Simple software systems, particularly ones that can be represented using regular expressions, can be modeled as Finite State Machines.
There are many of such simple systems, such as vending machines or basic electronics.

By finding the intersection of two Finite state machines, one can design in a very simple manner concurrent systems that exchange
messages for instance. For example, a traffic light is a system that consists of multiple subsystems, such as the different traffic lights, that
work concurrently.

Some examples of applications:

 number classification
 watch with timer
 vending machine
 traffic light
 bar code scanner
 gas pumps

Summary[edit]
Implementing a Moore or Mealy machine is not that hard, especially if one does not intend to minimize the circuit.

However, it does take practice to be able to do it reasonably fast, and is difficult to understand conceptually, at least, at first. One should trace
out a few steps just to convince oneself of how it behaves. There is a subtle difference in the behavior of the implementation of a Mealy and
Moore machine. Tracing out a few steps can show the differences more clearly.

Usually, it is harder to understand why the circuit does the right thing (i.e., implements the FSM) than it is to know how to build it. Certainly,
one should devote time to both understanding why this technique works, as well as mastering the technique.

Introduction to Grammars
In the literary sense of the term, grammars denote syntactical rules for
conversation in natural languages. Linguistics have attempted to define
grammars since the inception of natural languages like English, Sanskrit,
Mandarin, etc. The theory of formal languages finds its applicability
extensively in the fields of Computer Science. Noam Chomsky gave a
mathematical model of grammar in 1956 which is effective for writing
computer languages.

Grammar
A grammar G can be formally written as a 4-tuple (N, T, S, P) where

 N or VN is a set of Non-terminal symbols

 T or ∑ is a set of Terminal symbols

 S is the Start symbol, S ∈ N

 P is Production rules for Terminals and Non-terminals

Example
Grammar G1 −

({S, A, B}, {a, b}, S, {S → AB, A → a, B → b})

Here,
S, A, and B are Non-terminal symbols;

a and b are Terminal symbols

S is the Start symbol, S ∈ N

Productions, P : S → AB, A → a, B → b

Example
Grammar G2 −

({S, A}, {a, b}, S,{S → aAb, aA →aaAb, A → ε } )

Here,

 S and A are Non-terminal symbols.

 a and b are Terminal symbols.

 ε is an empty string.

 S is the Start symbol, S ∈ N

 Production P : S → aAb, aA → aaAb, A → ε

Derivations from a Grammar

Strings may be derived from other strings using the productions in a
grammar. If a grammarG has a production α → β, we can say that x α
y derives x β y in G. This derivation is written as −

x α y ⇒G x β y

Example
Let us consider the grammar −

G2 = ({S, A}, {a, b}, S, {S → aAb, aA → aaAb, A → ε } )

Some of the strings that can be derived are −

S ⇒ aAb using production S → aAb

⇒ aaAbb using production aA → aAb

⇒ aaaAbbb using production aA → aAb

⇒ aaabbb using production A → ε

Language Generated by a Grammar

The set of all strings that can be derived from a grammar is said to be the
language generated from that grammar. A language generated by a
grammar G is a subset formally defined by
 L(G)={W|W ∈ ∑*, S ⇒G W}

If L(G1) = L(G2), the Grammar G1 is equivalent to the Grammar G2.

Example
If there is a grammar

G: N = {S, A, B} T = {a, b} P = {S → AB, A → a, B → b}

Here S produces AB, and we can replace A by a, and B by b. Here, the only
accepted string isab, i.e.,

L(G) = {ab}

Example
Suppose we have the following grammar −

G: N={S, A, B} T= {a, b} P= {S → AB, A → aA|a, B → bB|b}

The language generated by this grammar −

L(G) = {ab, a2b, ab2, a2b2, ………}

Construction of a Grammar Generating a

Language
We’ll consider some languages and convert it into a grammar G which
produces those languages.

Example
Problem − Suppose, L (G) = {am bn | m ≥ 0 and n > 0}. We have to find out
the grammarG which produces L(G).

Solution

Since L(G) = {am bn | m ≥ 0 and n > 0}

the set of strings accepted can be rewritten as −

L(G) = {b, ab,bb, aab, abb, …….}

Here, the start symbol has to take at least one ‘b’ preceded by any number
of ‘a’ including null.

To accept the string set {b, ab,bb, aab, abb, …….}, we have taken the
productions −

S → aS , S → B, B → b and B → bB

S → B → b (Accepted)

S → B → bB → bb (Accepted)
S → aS → aB → ab (Accepted)

S → aS → aaS → aaB → aab(Accepted)

S → aS → aB → abB → abb (Accepted)

Thus, we can prove every single string in L(G) is accepted by the language
generated by the production set.

Hence the grammar −

G: ({S, A, B}, {a, b}, S, { S → aS | B, B → b | bB })

Example
Problem − Suppose, L (G) = {a m bn | m> 0 and n ≥ 0}. We have to find out
the grammar G which produces L(G).

Solution −

Since L(G) = {am bn | m> 0 and n ≥ 0}, the set of strings accepted can be
rewritten as −

L(G) = {a, aa, ab, aaa, aab ,abb, …….}

Here, the start symbol has to take at least one ‘a’ followed by any number
of ‘b’ including null.

To accept the string set {a, aa, ab, aaa, aab, abb, …….}, we have taken the
productions −

S → aA, A → aA , A → B, B → bB ,B → λ

S → aA → aB → a λ → a (Accepted)

S → aA → aaA → aaB → aaλ → aa (Accepted)

S → aA → aB → abB → abλ → ab (Accepted)

S → aA → aaA → aaaA → aaaB → aaa λ → aaa (Accepted)

S → aA → aaA → aaB → aabB → aab λ → aab (Accepted)

S → aA → aB → abB → abbB → abb λ → abb (Accepted)

Thus, we can prove every single string in L(G) is accepted by the language
generated by the production set.

Hence the grammar −

G: ({S, A, B}, {a, b}, S, {S → aA, A → aA | B, B → λ | bB })

 Chomsky Classification of Grammars
According to Noam Chomosky, there are four types of grammars − Type 0, Type 1, Type 2, and
Type 3. The following table shows how they differ from each other −

Grammar Type Grammar Accepted Language Accepted Automaton

Type 0 Unrestricted grammar Recursively enumerable Turing Machine

language

Type 1 Context-sensitive grammar Context-sensitive Linear-bounded

language automaton

Type 2 Context-free grammar Context-free language Pushdown

automaton

Type 3 Regular grammar Regular language Finite state

automaton

Take a look at the following illustration. It shows the scope of each type of
grammar −

Type - 3 Grammar
Type-3 grammars generate regular languages. Type-3 grammars must
have a single non-terminal on the left-hand side and a right-hand side
consisting of a single terminal or single terminal followed by a single non-
terminal.

The productions must be in the form X → a or X → aY

where X, Y ∈ N (Non terminal)

and a ∈ T (Terminal)

The rule S → ε is allowed if S does not appear on the right side of any rule.
Example

X → ε
X → a
X → aY

Type - 2 Grammar
Type-2 grammars generate context-free languages.

The productions must be in the form A → γ

where A ∈ N (Non terminal)

and γ ∈ (T∪N)* (String of terminals and non-terminals).

These languages generated by these grammars are be recognized by a non-

deterministic pushdown automaton.

Example

S → X a
X → a
X → aX
X → abc
X → ε

Type - 1 Grammar
Type-1 grammars generate context-sensitive languages. The productions
must be in the form

αAβ→αγβ

where A ∈ N (Non-terminal)

and α, β, γ ∈ (T ∪ N)* (Strings of terminals and non-terminals)

The strings α and β may be empty, but γ must be non-empty.

The rule S → ε is allowed if S does not appear on the right side of any rule.
The languages generated by these grammars are recognized by a linear
bounded automaton.

Example

AB → AbBc
A → bcA
B → b

Type - 0 Grammar
Type-0 grammars generate recursively enumerable languages. The
productions have no restrictions. They are any phase structure grammar
including all formal grammars.

They generate the languages that are recognized by a Turing machine.

The productions can be in the form of α → β where α is a string of terminals
and non-terminals with at least one non-terminal and α cannot be null. β is
a string of terminals and non-terminals.

Example

S → ACaB
Bc → acB
CB → DB
aD → Db