GENETIC ALGORITHMS

Contents
 1. Introduction
 2. Examples
 3. Genetic Algorithms & Applications
 4. Conclusion
 1. Introduction
What is GA

 Genetic algorithm (or GA)

 Search technique used in computing to find true or approximate solutions to
optimization and search problems.

 search-based optimization technique based on the principles of Genetics and Natural

Selection.
 It is frequently used to find optimal or near-optimal solutions to difficult problems which
otherwise would take a lifetime to solve.

 Genetic algorithms are a particular class of evolutionary algorithms

that use techniques which are
 Inspired by evolutionary biology such as inheritance, mutation, selection, and crossover
(also called recombination).
Introduction (Contd.)
Why GA
 Most real world optimization problems involve complexities like discrete, continuous or mixed variables,
multiple conflicting objectives, non-linearity, discontinuity and non-convex region.

 The search space (design space) may be so large that global optimum cannot be found in a reasonable time.

 The existing linear or nonlinear methods may not be efficient or computationally expensive for solving such
problems.

 Various stochastic search methods like Simulated annealing, Evolutionary Algorithms (EA) or Hill Climbing can
be used in such situations.

 EAs have the advantage of being applicable to any combination of complexities (multi-objective, non-linearity
etc) and also can be combined with any existing local search or other methods.

 Various techniques which make use of EA approach are

 Genetic Algorithms (GA), Ant Bee Colony algorithm, Particle Swarm Optimisation algorithm etc.

 All these EA techniques operate mainly on a population search basis.

 Genetic Algorithms is one of the most popular EA techniques

History of GA’s
 Directed search algorithms based on the mechanics of biological
evolution
 Developed By John Holland, University of Michigan (1970)
 His aim was to make computers do what nature does.

 To understand
 The adaptive processes of natural systems
 To design artificial systems software that retains
 The robustness of natural systems

 Holland was concerned with algorithms that manipulate strings of binary

digits.
1 0 1 1 0 1 0 0 0 0 0 1 0 1 0 1

5
 Classes of Search Techniques

Search
Techniques
Calculations Enumerative
BasedTechniqes Guided random search Techniqes
techniques

Fibonacci Sort DFS Dynamic BFS

Programming

Tabu Search Hill Simulated Evolutionary

Climbing Annealing Algorithms

Ant Colony Genetic

Optimization Algorithms
Etc

6
 Basic Terminology
 Search Space
 All possible solutions to the problem

 Population
 It is a subset of all the possible (encoded) solutions to the given problem.

 Chromosomes
 A chromosome is one such solution to the given problem.

 Gene
 A gene is one element position of a chromosome.

 Genome
 Collection of all chromosomes for an individual

 Allele
 It is the value a gene takes for a particular chromosome.

 Individual
 Any possible solution

 Trait
 Possible aspect (features) of an individual

 Locus
 The position of a gene on the chromosome
 Genotype – Genotype is the population in the computation
space. In the computation space, the solutions are represented in
a way which can be easily understood and manipulated using a
computing system.

 Phenotype – Phenotype is the population in the actual real

world solution space in which solutions are represented in a way
they are represented in real world situations.

 Fitness Function – A fitness function simply defined is a

function which takes the solution as input and produces the
suitability of the solution as the output.
 In some cases, the fitness function and the objective function may be the
same, while in others it might be different based on the problem.

 Genetic Operators – These alter the genetic composition of the

offspring. These include crossover, mutation, selection, etc.
 Basic steps in GA algorithms

1. Initialization
1. Initially many individual solutions are randomly generated to form an initial population. The
population size depends on the nature of the problem, but typically contains several hundreds or
thousands of possible solutions.

2. Selection
1. A proportion of the existing population is selected to breed a new generation.
2. Individual solutions are selected through a fitness-based process, where fitter solutions (as measured
by a fitness function) are typically more likely to be selected. (Ex: Roulette wheel selection)

3. Reproduction
1. Generate a second generation population of solutions from those selected through genetic
operators:
1. Crossover (also called recombination)
2. Mutation

4. Termination
1. This generational process is repeated until a termination condition has been
reached.
 Common terminating conditions are:
 A solution is found that satisfies minimum criteria
 Cross Over and Mutation

 Cross Over
 In single point crossover, you choose a locus at which you swap the remaining alleles from
one parent to the other. This is complex and is best understood visually.
 Example: As you can see, the children take one section of the chromosome from each
parent.
 The probability of crossover occurring is usually 60% to 70%.

 Mutation
 Mutation is fairly simple. You just change the selected alleles based on what you feel is
necessary and move on.
 Mutation is, however, vital to ensuring genetic diversity within the population.
 Psudo Code of basic Genetic Algorithm

Algorithm

Initialize population;

Evaluate population;

while Termination_Criteria_Not_Satisfied

Select parents for reproduction;

Perform recombination and mutation;

Evaluate population;

}
Genetic Algorithm (GA) Functioning

Generation Generation
0 1
Individuals Fitness Offspring

011 $3 111
001 $1 010
110 $6 110
010 $2 010
 Detailed Explanation: Steps in GA

 Step 1:
 Represent the problem variable domain as a chromosome of a fixed length,
choose the size of a chromosome population N, the crossover probability pc and
the mutation probability pm.
 Step 2:
 Define a fitness function to measure the performance, or fitness, of an individual
chromosome in the problem domain. The fitness function establishes the basis
for selecting chromosomes that will be mated during reproduction.
 Step 3:
 Randomly generate an initial population of chromosomes of size N:
x 1, x 2 , . . . , x N
 Step 4:
 Calculate the fitness of each individual chromosome:
f (x1), f (x2), . . . , f (xN)
 Step 5:
 Select a pair of chromosomes for mating from the current population.
Parent chromosomes are selected with a probability
related to their fitness.
Selection of a pair of Parents

 One Pair at a time

 Rowlett Wheel is a popular procedure
 Step 6:
 Create a pair of offspring chromosomes by applying the genetic operators
 Crossover and Mutation.
 Step 7:
 Place the created offspring chromosomes in the new population.
 Step 8:
 Repeat Step 5 until the size of the new chromosome population becomes equal to the
size of the initial population, N.
 Step 9:
 Replace the initial chromosome population with the fittest chromosomes (which are
having higher fitness value). The new population may contain part of new generation as
well as part of previous generation.
 Step 10:
 Go to Step 4, and repeat the process until the termination criterion is satisfied.
 Example - 1

A Simple Example: The Traveling Salesman Problem:

Find a tour of a given set of cities so that

 Each city is visited only once
 The total distance traveled is minimized

 Starting city may not be same as ending city (one example problem)
 Starting city and ending city are the same (another example problem)

17
Representation
Representation is an ordered list of city [numbers known as an order-based
GA]
1) London 3) Dunedin 5) Beijing 7) Tokyo
2) Venice 4) Singapore 6) Phoenix 8) Victoria
-------------------------------------------------------

City List 1 (3 5 7 2 1 6 4 8)
City List 2 (2 5 7 6 8 1 3 4)

18
 Crossover
Crossover combines inversion and recombination (locus
points from 3 to 6 have been selected for cross-over)
City List-1 (3 5 7 2 1 6 4 8)
City List-2 (2 5 7 6 8 1 3 4)
City List- New1 (2 5 7 8 1 6 3 4)
City List- New2 (3 5 7 6 2 1 4 8)
This operator is called order-based crossover.
After the cross-over, a minor correction is required to fulfil
the condition of uniqueness of indices. Changed indices are
shown in red color
19
Mutation
Mutation applied on first off-spring [City List- New1]. Here as
part of mutations, swapping is performed (we need to define
mutation, contextually). 2nd and 7th locations have been
swapped.

Before: (2 5 7 8 1 6 3 4)

After: (2 3 7 8 1 6 5 4)

20
New Population after cross-over and mutation
(8 cities case)

1. City List-1 (3 5 7 2 1 6 4 8)
2. City List-2 (2 5 7 6 8 1 3 4)
3. City List-3 (2 3 7 8 1 6 5 4) child-1
4. City List-4 (3 5 7 6 2 1 4 8) child-2
Fitness value of each list is computed:
Definition of Fitness function vary problem to problem
Fitness function for TSP problem?
Retain the lists with fitness values more than the pre-defined
threshold
21
TSP Example: 30 Cities

22
Solution i (Distance = 941)

23
Solution j (Distance = 800)

24
Solution k (Distance = 652)

25
Best Solution (Distance = 420)

26
Overview of Performance

27
 Example: f(x)=x2

1.Selection

Selected for cross over

A.E Eiben and J.E Smith, Introduction to Evolutionary Computing :Genetic Algorithms
 2.Crossover

Selected for mutation

A.E Eiben and J.E Smith, Introduction to Evolutionary Computing :Genetic Algorithms
3.Mutation

A.E Eiben and J.E Smith, Introduction to Evolutionary Computing :Genetic Algorithms
Considering the GA Technology

“Almost eight years ago ...

people at Microsoft wrote a program [that] uses some
genetic things for finding short code sequences.
Windows 2.0 and 3.2, NT, and almost all Microsoft
applications products have been incorporated with this
pieces of code created by that system.”
- Nathan Myhrvold, Microsoft Advanced Technology Group, Wired, September 1995
3. Genetic Algorithms –Applications

Some Application Types: GA

Domain Application Types
Control gas pipeline, pole balancing, missile evasion, pursuit

Design semiconductor layout, aircraft design, keyboard

configuration, communication networks
Scheduling manufacturing, facility scheduling, resource allocation

Robotics trajectory planning

Machine Learning designing neural networks, improving classification

algorithms, classifier systems
Signal Processing filter design

Game Playing poker, checkers, prisoner’s dilemma

Combinatorial set covering, travelling salesman, routing, bin packing,

graph colouring and partitioning
Optimization
When to Use a GA
 Alternate solutions are too slow or overly complicated
 Need an exploratory tool to examine new approaches
 Problem is similar to one that has already been successfully solved
by using a GA
 Want to hybridize with an existing solution
 Benefits of the GA technology meet key problem requirements
Issues for GA Practitioners
 Choosing basic implementation issues:
 Representation

 Population Size and Mutation rate

 Selection and Deletion policies

 Crossover and Mutation operators

 Termination Criteria

 Performance, scalability

 Solution is only as good as the evaluation function (often hardest part)

Pros and Cons :GA
 Pros
 Faster (and lower memory requirements) than searching a very
large search space.
 Easy, in that if your candidate representation and fitness function
are correct, a solution can be found without any explicit analytical
work.

 Cons
 Randomized – not optimal or even complete.
 Can get stuck on local maxima, though crossover / mutation can
help mitigate this.
 It can be hard to work out how best to represent a candidate as a
bit string (or otherwise).
35
Local Minima Problem
Benefits of Genetic Algorithms
 Concept is easy to understand

 Modular, separate from application

 Supports multi-objective optimization

 Good for “noisy” environments

 Always an answer; answer gets better with time

 Inherently parallel; easily distributed

 Easy to exploit previous or alternate solutions

 Flexible building blocks for hybrid applications

 Substantial history and range of use

 4. Conclusion

 Genetic algorithms are original systems based on the

supposed functioning of the living.

 The GA method is different from classical optimization

algorithms.

 These algorithms are extremely efficient, and are used in

many fields
 References

1. C. Darwin. On the Origin of Species by Means of Natural Selection; or, the

Preservation of flavored Races in the Struggle for Life. John Murray, London, 1859.

2. W. D. Hillis. Co-Evolving Parasites Improve Simulated Evolution as an Optimization

Procedure. Artificial Life 2, vol 10, Addison-Wesley, 1991.

3. E. Anderson. The Irises of the Gaspe Peninsula. Bull. Amer. Iris Soc., 59:2-5, 1935.

4. Bäck and H.-P. Schwefel. An overview of evolutionary algorithms for par greeter
optimization. Evolutionary Computation, 1(1):1-23, 1993.

5. Book: Genetic Algorithms + Data Structures = Evolution Programs, Michalewicz

Zbigniew

6. Book: Practical Handbook of Genetic Algorithms (in 3 volumes), Lance Chambers

D.

7. Book: Genetic algorithms reference (in 2 volumes), T.D.Gwiazda

8. Book: Genetic Algorithms in Search, Optimization, and Machine

Learning, Goldberg D.E.
“Questions”
?