Basic concepts of Genetic

Algorithm (GA)

Jayeeta Mitra
Associate Professor
AGFE, IIT Kharagpur
• Gradient descent method is an iterative first order
optimization algorithm to find a local maxima/minima of a
given function. Commonly used in Machine learning (ML) and
Deep Learning (DL).
• Due to its importance and ease of implementation it is used
commonly in many applications including control engineering,
chemical, mechanical robotics etc.
• To use this function has to be
E = f (v, w); Δv = -η ; Δw = - η
 differentiable and
 convex
•
 Optimisation
• Genetic algorithm is a population based probabilistic search
and optimization technique works on Darwin’s principle of
natural selection.

• Genetic algorithms (GAs)

are search and
optimization tools, which
work differently
compared to classical
search and optimization
methods
• Because of their population approach, they have also
been extended to solve other search and optimization
problems efficiently, including multimodal, multi-
objective and scheduling problems, as well as fuzzy-GA
and neuro-GA implementations.
 GENETIC ALGORITHM BASED OPTIMIZATION
• The GA was introduced by Prof. Jhon Holland,
University of Michigan, Ann Arbor, USA 1995.
• It is basically an iterative search technique working
based on the concept of probability.
• Genetic algorithm is based on the natural rules
associated with procreation, where basic information
is passed from generation to generation through coded
DNA molecules.
 GENETIC ALGORITHM BASED
OPTIMIZATION
• Maximum or minimum values of fitness
function reflecting the entire range of
independent variables are arrived at by this
algorithm which involves reproduction,
crossover, mutation and identification of
penalty function.
 Dependent and independent variables
parameters
X = Independent variable
Y = Dependent variable
x = Coded independent variable valued between -1
and +1
y = Coded dependent variable valued between 0 and
+1
Xmax = Maximum value of X
Xmin = Minimum value of Y
 Contd….
For independent variables
XM = (Xmax + Xmin)/2
XD = Xmax - XM
x = (X – XM)/XD
X = xXD + XM
For dependent variables
YM = (Ymax + Yminp)/2
Yminp = 2Ymin - Ymax
YD = Ymax - YM
y = (Y - YM)/YD
Y = yYD + YM
 Binary value calculation
Representing independent variable into an array or string
of binary digits (bits)of 0 and 1. binary string is similar to
a biological chromosome. Every bit represents a
particular traits (gene vlue).
10101 = 1x24+0x23+1x22+0x21+1x20=21
vb = u1 2m – 1 + u2 2m – 2 + u3 2 m – 3 + ………..+ um 20
U1, 2, 3……….. Binary digit 1 or 0
1111=1x23+1x22+1x21+1x20 = 15= 24-1
0000=0x23+0x22+0x21+0x20 = 0
• Length of a string decided based on precision applied (ap)
in the value of variables.
 Contd….
𝑋𝑋𝑚𝑚𝑚𝑚𝑚𝑚 − 𝑋𝑋𝑚𝑚𝑚𝑚𝑚𝑚 2𝑚𝑚 − 1
=
2𝑎𝑎𝑝𝑝 1

If vb is the binary value of one of the segmented

population string of length m (m can take the value f m1,
m2 and m3……) and X is the corresponding real value of
string, the relationship between vb and X is given by
𝑋𝑋𝑚𝑚𝑚𝑚𝑚𝑚 − 𝑋𝑋𝑚𝑚𝑚𝑚𝑚𝑚 𝑋𝑋 − 𝑋𝑋𝑚𝑚𝑚𝑚𝑚𝑚
𝑚𝑚
=
2 −1 𝑣𝑣𝑏𝑏

M = m1 + m2 + m3 +….
N = 1.65 + 20.21 M
 Steps in GA
• Random generation of initial population string.
• calculate fitness value (objective function in case
of maximisation problem)of each solution in the
population.
• Let f(x) to be minimised, then we can have
maximisation objective function by the
following:
• Maximise –f(x) using duality principle
• Maximise 1/f(x), for f(x)≠0
• Maximise 1/(1+f(x)), for f(x) ≥ 0 etc.
 GA Parameters
1Generation of GA Reproduction/selection

Fitness value
Crossover

Mutation

• Population of solution is then modified using different operators

namely reproduction, crossover, mutation and others.
• The primary objective of the selection operator is to emphasize the
good solutions and eliminate the bad solutions in a population while
keeping the population size constant.
• A fitness value is assigned to each solution depending on how close it
is actually to the optimal solution of the problem
 Steps in GA

• Reproduction (selection scheme)-of good solution using their

fitness value. Proportionate selection-Roulette-wheel selection,
tournament selection, ranking selection etc.

• Crossover(exchange of property between the parents)-mating

pairs are selected at random for crossover based on crossover
probability (0.1 to 1). Single point, two point, multipoint,
uniform crossover.

• Mutation (sudden change of parameters)-achieving a local

change around the current solution.
 Reproduction
• Proportionate selection-Roulette-wheel selection:
• The probability of a string for being selected in the mating pool
is proportional to its fitness value.
• Probability of selection of ith area will be:
𝑓𝑓
• 𝑝𝑝 = ∑𝑁𝑁 𝑖𝑖
𝑖𝑖=1 𝑓𝑓𝑖𝑖

• The wheel will be rotated for N times

• Good string will be selected more than once.
 Reproduction
• A GA search depends on the factors like population diversity
(exploration) and selection pressure (exploitation)

• A higher selection pressure will leads to loss of diversity and

premature convergence of a solution to a sub optimal value.

• A lower selection pressure may not be able to drive search

properly and consequently a stagnation may occur.

• To overcome this rank based selection is proposed.

 Reproduction

• Ranking selection:
• Let us assume 4 population-4 fitness values f1, f2, f3 and f4
contributes 80%, 10%, 7% and 3% of the total fitness
value.
• The strings are arranged in ascending order of their
fitness value. – f4, f3, f2, f1.
• Rank them. f4-1, f3-2, f2-3, f1-4
• Then proportionate selection scheme based on the
assigned rank is adopted.
 Reproduction
• The % area tobe occupied by a particular string (ith ) is given
by the expression (Ri/∑R) x 100, Where Ri is the rank of ith
string.

• Thus the string f1, f2, f3 and f4 will occupy 40%, 30%, 20% and
10% of the total area respectively and consequently their
probabilities or being selected in the mating pool become
equal to 0.4, 0.3, 0.2 and 0.1. rank based proportionate
selection scheme is better than the fitness based proportionate
selection
 Reproduction
• Tournament selection:
• N tournaments were played between the n population.
Each string selected for tournament twice but the
probability of their win may be o, 1 or two times based
on their fitness.

• It is computationally faster than both the fitness-based

and rank based proportionate selection scheme.

• Elitism-direct copy of the good strings

 Crossover
• Crossover probability says how often will be crossover
performed. If there is no crossover, offspring is exact copy of
parents. If there is a crossover, offspring is made from parts of
parents' chromosome.

• If crossover probability is 100%, then all offspring is made by

crossover. If it is 0%, whole new generation is made from
exact copies of chromosomes from old population (but this
does not mean that the new generation is the same!).

• Crossover is made in hope that new chromosomes will have

good parts of old chromosomes and maybe the new
chromosomes will be better.
One-point, two-point, and uniform crossover methods
 Mutation
• It is done to bring a local change over the current solution. 1 is
converted to 0 and vice versa.

• The plot of objective function

having one local basin and
global basin.
• Let us suppose that all the
randomly generated initial
solution of GA fall on the local
basin by chance.
• Mutation probability range in
between 0.1/L to 1/L, where L
is the string length.
• GA generation stops unless Role of mutation
 GA parameter setting
• The performance of a genetic search depends on the amount of
exploration ( population diversity) and exploitation (selection
pressure).
• To have an effective search there must be a proper balance
between them and to ensure this the GA parameters such as
population size N, crossover probability pc, mutation probability
pm are to be selected in the optimal sence.
• The interaction among themselves are to be studied to elect
their optimal values.
• It is obvious that the optimal GA parameters are problem
dependent.
 A method of determining a near-optimal GA-
parameters has been discussed
• a minimization problem:
• Vary GA parameters such as crossover
probability pc, mutation probability
pm, population size N, number of
generations Gmax, will be varied in the
range of (0.6 to 1), (0.001 to 0.011),
(50 to150) and (50 to 150).
• Stage 1: pc (0.6 to 1, with a step of
0.04) keeping pm, N and G at their
mid-value. Minimum fitness at pc = 0.9
• Stage 2: fix pc at 0.9, pm (0.001 to
0.011 with a step 0.001), Minimum
fitness at pm = 0.002
• Stage 3: fix pc = 0.9, pm =0.002,
N (50 to150 with a step of 10)
and G at their mid-value.
Minimum fitness at N=110
• Stage 4: fix pc = 0.9, pm =0.002,
N =110, vary G(50 to 150 with a
step of 10). Minimum fitness is
obtained at 120 generations.
• Final GA parameters are found
to be pc = 0.9, pm =0.002, N
=110 and G=120.
Example:

Maltodextrin (X1), Glycerol mono – stearate (X2), and

tricalcium phosphate (X3) - Independent variables
Sticky point temperature (Y1), and Flow time (Y2) – Dependent
variables
X1 max = 0.62; X2 max = 0.020; X3 max = 0.020
X1 min = 0.28; X2 min = 0.010; X3 min = 0.010
ap1 = 0.005; ap2 = 0.0005; ap3 = 0.0005
m1 = 6; m2 = 4; m3 = 4 𝑋𝑋𝑚𝑚𝑚𝑚𝑚𝑚 − 𝑋𝑋𝑚𝑚𝑚𝑚𝑚𝑚 2𝑚𝑚 − 1
M = m1 + m2 + m3 = 14 =
2𝑎𝑎𝑝𝑝 1
N = 1.65 + 20.21 M = 9.324 ≈ 10
 Calculation Steps

1. Identify Xi max, Xi min, Yj max, Yj min, XiM, XiD, YjM, YjD:

XiM = (Xi max + Xi min)/2
XiD = Xi - XiM
YjM =(Yj max + Yj minp)/2
Yj minp = 2Yj min - Yj max
YjD = Yj max – YjM
2. Convert to coded variables:
xi = (Xi - XiM)/XiD
yj = (Yj - YjM)/YjD
 Steps: contd…..

3. Identify string length m from Xi max, Xi min, and vb

Vb = u1 2m – 1 + u2 2m – 2 + u3 2 m – 3 + ………..+ um 20
2m – 1 = (Xi max - Xi min)/(2ap)
ap is the precision
2m = (Xi max - Xi min)/(2ap) + 1
m loge 2 = loge ((Xi max - Xi min)/(2ap) + 1)
m = loge ((Xi max - Xi min)/(2ap) + 1)/loge 2
ap = (Xi max - Xi min)/(2(2m - 1))

Vb =
 Steps……contd.

4. M = ;N

5. Generate N x M binary string from the whole

population.
6. Obtain corresponding
Vb = u1 2m – 1 + u2 2m – 2 + u3 2 m – 3 + ………..+ um 20
7. Xi = Xmin + Vb x
 Steps…..contd.

8. xi = (Xi -XM)/XD
9. Yj = f (xi) =
 Steps…….contd.

10. yj = (Yj - YjM)/YjD

11. Fitness function F = yj + 1/(1 + yj+1). Compute average F
value.
12. Random number generation for tournament
participation.
13. Select participants on elitism.
14. Obtain new F values. Compute average F value.
15. Select crossover pairs by random number generation.
16. Select crossover sites by random number and
mismatch sites.
 Steps….concl.

17. Identify new population.

18. Repeat steps 6 – 11.
19. Select N best strings on the basis of mixing F values
obtained on steps 14 and 18. Obtain average F value.
20. Carry out mutation on only one string by random number
generation.
21. Repeat steps 6 – 11 for mutated string.
22. Obtain new F value after replacing the pre-mutated F value.
23. Compute average F value and observe improvement over
the whole process.
 Constraints optimization in GA

• Constraints may be linear, non linear, equality, inequality and

others which basically divides the search space into feasible
and infeasible regions.
• A constrained optimization (either maximization and
minimization ) problem may be expressed as follows.
• Optimize f(X)
• Subject to
• gi(X) ≤ 0, i =1,2, …..n (n- inequality constraints)
• Hj(X) = 0, i =1,2, …..p, (p- equality constraints)
 Constraints optimization in GA
• X= (x1, x2, x3…..xm)T are the design variables having the
bounds given below.

• 𝑥𝑥1𝑚𝑚𝑚𝑚𝑚𝑚 ≤ 𝑥𝑥1 ≤ 𝑥𝑥1𝑚𝑚𝑚𝑚𝑚𝑚

• 𝑥𝑥2𝑚𝑚𝑚𝑚𝑚𝑚 ≤ 𝑥𝑥2 ≤ 𝑥𝑥2𝑚𝑚𝑚𝑚𝑚𝑚
• …………………………..
𝑚𝑚𝑚𝑚𝑚𝑚 𝑚𝑚𝑚𝑚𝑚𝑚
• 𝑥𝑥𝑚𝑚 ≤ 𝑥𝑥𝑚𝑚 ≤ 𝑥𝑥𝑚𝑚

• There are many methods available for constrained

optimization. One is penalty function approach.
 Penalty Function Approach.
• The fitness function of a solution (say ith) is expressed by
modifying its objective function as follows:
• 𝐹𝐹𝑖𝑖 (𝑋𝑋) = 𝑓𝑓𝑖𝑖 (𝑋𝑋) ± 𝑃𝑃𝑖𝑖
• Where Pi indicates the penalty used to penalize an infeasible
solution. For a feasible solution, Pi is set equal to 0.0. whereas
for an infeasible solution Pi is expressed like the following:
𝑞𝑞 2
• 𝑃𝑃𝑖𝑖 = 𝐶𝐶 ∑𝑘𝑘=1 ∅𝑖𝑖𝑖𝑖 (𝑋𝑋)
• Where, C indicates the user defined penalty coefficient and
∅𝑖𝑖𝑖𝑖 (𝑋𝑋) represents the penalty term for kth constraints
corresponding to the ith objective function.
• Penalty could be static, dynamic, and adaptive.
 Problems
• Maximize y= (x)(1/2)
• Subject to 1.0 ≤ x ≤16.0
• Use Roulette wheel selection
• Single point crossover with pc=1.0 and bitwise
mutation pm=0.03
 References
• D. E. Goldberg, ‘Genetic Algorithm In Search, Optimization
And Machine Learning’, New York: Addison –Wesley (1989)

• John H. Holland ‘Genetic Algorithms’, Scientific American

Journal, July 1992.

• Kalyanmoy Deb, ‘An Introduction To Genetic Algorithms’,

Sadhana, Vol. 24 Parts 4 And 5.

• D K Pratihar, ‘soft computing fundamentals and

application’,Narosa Publication House.
 A simple case of GA
• The working principle of GAs is very different from that of most
of classical optimization techniques.
• We describe the working of a GA by illustrating a simple can
design problem.
• A cylindrical can is considered to have only two design
parameters - diameter d and height h.
• Let us consider that the can needs to have a volume of at least
300 ml and the objective of the design is to minimize the cost
of can material.
• With these constraints and objective, we first write the
corresponding nonlinear programming problem (NLP).
 corresponding nonlinear programming problem (NLP)

• The parameter c is the cost of can material per square cm, and
diameter d and height h are allowed to vary in [dmin, dmax] and
[hmin, hmax] cm, respectively.
• Represent the parameter values in binary strings.
• Let us assume that we shall use five bits to code each of the two
design parameters d and h, thereby making the overall string
length equal to 10.
• The following string represents a can of diameter 8 cm and height 10
cm,

• The lower and upper bounds of both parameters are considered

to be zero and 31, respectively.
• With five bits to represent a parameter, there are exactly 25 or 32
different solutions possible.
• Choosing the lower and upper bounds as above allows GAs to
consider only integer values in the range [0, 31].
• However, GAs are not restricted to using only integer values
in the above range. In fact GAs can be assigned to use any
other integer or non-integer values just by changing the
string length and lower and upper bounds.
• To see how these 10 genes control the phenotype (the shape) of the
can, let us investigate the leftmost bit (gene) of the diameter (d)
parameter.

• A value of 0 at this bit (the most significant bit) allows the can to have
diameter values in the range [0, 15] cm, whereas the other value 1
allows the can to have diameter values in the range [16, 31] cm.

• Clearly, this bit (or gene) is responsible for dictating the slimness of the
can

• Each bit position or combination of two or more bit positions can also
be explained to have some feature of the can, but some are interesting
and important and some are not that important.
• Now that we have achieved a string representation of design
solution, we are ready to apply some genetic operations to such
strings to hopefully find better and better solutions.
• Assigning fitness to a solution: In most cases, however, the fitness
is made equal to the objective function value. For example, the
fitness of the above can be represented by the 10-bit string is

• assuming c = 0.0654. Since the objective of the optimization

is to minimize the objective function, it is to be noted that a
solution with a smaller fitness value is better compared to
another solution.
Reproduction
• Population after reproduction
 crossover

• single-point crossover operator

• Moreover, every crossover may not create better solutions,

but we do not worry about it too much. If bad solutions are
created, they get eliminated in the next reproduction
operator and hence have a short life.
 Mutation

• The crossover operator is mainly responsible

for the search aspect of genetic algorithms,
even though the mutation operator is also
used for this purpose sparingly.
• We use five-bit strings to represent the variable x in the range
[0, pi], so that the string (00000) represents x = 0 solution and
the string (11111) represents x = pi solution.
• Other 30 strings are mapped in the range [0, pi].
• Let us also assume that we shall use a population of size four,
proportionate selection 3, single-point crossover with
probability one, and bit-wise mutation with a probability 0.01.