Q. Dynamic programming?

A. Dynamic programming amounts to breaking down an optimization problem into simpler sub-
problems, and storing the solution to each sub-problem so that each sub-problem is only solved
once. There will be certain times when we have to make a decision which affects the state of the
system, which may or may not be known to us in advance. These decisions or changes are equivalent to
transformations of state variables. The results of the previous decisions help us in choosing the future
ones.

What do we conclude from this? We need to break up a problem into a series of overlapping sub-
problems, and build up solutions to larger and larger sub-problems. If you are given a problem, which
can be broken down into smaller sub-problems, and these smaller sub-problems can still be broken into
smaller ones - and if you manage to find out that there are some over-lappping sub-problems, then
you've encountered a DP problem.

Some famous Dynamic Programming algorithms are:

 Unix diff for comparing two files

 Bellman-Ford for shortest path routing in networks
 TeX the ancestor of LaTeX
 WASP - Winning and Score Predictor

The core idea of Dynamic Programming is to avoid repeated work by remembering partial results and
this concept finds it application in a lot of real life situations.

In programming, Dynamic Programming is a powerful technique that allows one to solve different types
of problems in time O(n2) or O(n3) for which a naive approach would take exponential time.

Q. Particle swarm optimization ?

A. Particle swarm optimization (PSO) is a population based stochastic optimization technique developed
by Dr. Eberhart and Dr. Kennedy in 1995, inspired by social behavior of bird flocking or fish schooling.

PSO shares many similarities with evolutionary computation techniques such as Genetic Algorithms
(GA). The system is initialized with a population of random solutions and searches for optima by
updating generations. However, unlike GA, PSO has no evolution operators such as crossover and
mutation. In PSO, the potential solutions, called particles, fly through the problem space by following
the current optimum particles. The detailed information will be given in following sections.

Compared to GA, the advantages of PSO are that PSO is easy to implement and there are few
parameters to adjust. PSO has been successfully applied in many areas: function optimization, artificial
neural network training, fuzzy system control, and other areas where GA can be applied.

The remaining of the report includes six sections:

Background: artificial life.

The Algorithm
Comparisons between Genetic algorithm and PSO
Artificial neural network and PSO
PSO parameter control
Online resources of PSO

Q. Crossover and mutation operator in GA?

A. Crossover and mutation are two basic operators of GA. Performance of GA very depends on
them. Type and implementation of operators depends on encoding and also on a problem.

There are many ways how to do crossover and mutation. In this chapter are only some examples
and suggestions how to do it for several encoding.

Binary Encoding

Crossover

Single point crossover - one crossover point is selected, binary string from beginning of chromosome to
the crossover point is copied from one parent, the rest is copied from the second parent

11001011+11011111 = 11001111

Two point crossover - two crossover point are selected, binary string from beginning of
chromosome to the first crossover point is copied from one parent, the part from the first to the
second crossover point is copied from the second parent and the rest is copied from the first
parent

11001011 + 11011111 = 11011111

Uniform crossover - bits are randomly copied from the first or from the second parent
 11001011 + 11011101 = 11011111

Arithmetic crossover - some arithmetic operation is performed to make a new offspring

11001011 + 11011111 = 11001001 (AND)

Mutation

Bit inversion - selected bits are inverted

11001001 => 10001001

Permutation Encoding

Crossover

Single point crossover - one crossover point is selected, till this point the permutation is copied from the
first parent, then the second parent is scanned and if the number is not yet in the offspring it is added
Note: there are more ways how to produce the rest after crossover point

(1 2 3 4 5 6 7 8 9) + (4 5 3 6 8 9 7 2 1) = (1 2 3 4 5 6 8 9 7)

Mutation

Order changing - two numbers are selected and exchanged

 (1 2 3 4 5 6 8 9 7) => (1 8 3 4 5 6 2 9 7)

Value Encoding

Crossover

All crossovers from binary encoding can be used

Mutation

Adding a small number (for real value encoding) - to selected values is added (or subtracted) a small
number

(1.29 5.68 2.86 4.11 5.55) => (1.29 5.68 2.73 4.22 5.55)

Tree Encoding

Crossover

Tree crossover - in both parent one crossover point is selected, parents are divided in that point and
exchange part below crossover point to produce new offspring

Mutation

Changing operator, number - selected nodes are changed

Q. Fuzzy Reasoning with single and multiple antecedents?

A. Multiple antecedents - Consider the fuzzy mathematical programming (FMP) problem g(˜c, x) →
˜max (1) f1(˜a1, x) ≤ ˜b1 . . . fm(˜am, x) ≤ ˜bm where ˜c = (˜c1, . . . , c˜k) and ˜ai = (˜ai1, . . . , a˜in) are
vectors of fuzzy quantities (i.e. fuzzy sets of the real line IR), ˜bi is a fuzzyquantity, x = (x1, . . . , xn) is a
vector of decision variables, g(˜c, x) and fi(˜ai , x) are defined by Zadeh’s extension principle, and the
inequality relation ≤ is defined by a certain fuzzy relation. Now, by using FMR techniques, we shall
determine a fuzzy quantity, ˜max, which satisfies the inequality g(˜c, x) ≤ ˜max, ∀x ∈ IRn , under the
constraints fi(˜ai , x) ≤ ˜bi , i = 1, . . . , m. We consider the inequality relation ≤ as a fuzzy relation on IR
and for every x ∈ IRn determine ˜maxx from the following MFR scheme Antecedent 1: f1(˜a1, x) ≤ ˜b1 . .
. Antecedent m: fm(˜am, x) ≤ ˜bm Fact: g(˜c, x) Consequence: ˜maxx where, according to the MFR
inference rule, ˜maxx = g(˜c, x) ◦ min i=1,...,m (fi(˜ai , x) ≤ ˜bi). It is clear that ˜maxx is a fuzzy set
realizing the inequality g(˜c, x) ≤ ˜maxx under the premises fi(˜ai , x) ≤ ˜bi , i = 1, . . . , m. Finally, we
define the solution, ˜max, of the FMP problem (P1) as ˜max = sup x g(˜c, x) ◦ min i=1,...,m (fi(˜ai , x) ≤
˜bi) It is clear that ˜max is a fuzzy set realizing the inequality g(˜c, x) ≤ ˜max, ∀x ∈ X, under the premises
fi(˜ai , x) ≤ ˜bi , i = 1, . . . , m.

Single-antecedent fuzzy rules - in FKM We observe that the main limitation of having a map that
implements the original fuzzy theory is due to the difficulty in resolving the problem of implementing
multi-antecedent fuzzy rules in the arcs of an FCM. Take a hypothetical example: Suppose we have an
FCM as in Figure 4. A set of traditional fuzzy rules may be as follows: R1: If a is A1 and b is B1 then y is C1
R2: If a is A2 and b is B2 then y is C2 R3: If a is A3 and b is B3 then y is C3v
An FCM with two antecedents A and B, and a consequent C. With such a set of fuzzy rules, spreading the
rules over the two arcs is problematic. For example, we may have to decide where the rules should
reside – in arc A--C or arc B--C, or somewhere outside the FCM. As more nodes are added, one can
image the complexity of the relationships will become. During simulation, we may have to search
through all the rules in order to determine which rules get fired. This is exactly the problem of the
conventional fuzzy systems we are trying to avoid. RBFCM is an attempt to solve this problem by
providing only a single membership function in each arc. However there are many problems needed to
be ironed out. Reference [12] tried to solve these shortcomings by implementing fuzzy carry
accumulation (FCA) and shift operation in the computation of the outputs. For example in our example
as in Figure 4, we may have two fuzzy rules: • If A decreases then C increases; • If B increases then C
increases

Q. Dual Simplex Method?

A. DUAL SIMPLEX METHOD. In dual simplex method, the LP starts with an optimum (or better)
objective function value which is infeasible. Iterations are designed to move toward feasibility without
violating optimality. At the iteration when feasibility is restored, the algorithm ends.

Q. Architecture of RBFN?

A. the RBFN approach is more intuitive than the MLP. An RBFN performs classification by measuring the
input’s similarity to examples from the training set. Each RBFN neuron stores a “prototype”, which is just
one of the examples from the training set. When we want to classify a new input, each neuron
computes the Euclidean distance between the input and its prototype. Roughly speaking, if the input
more closely resembles the class A prototypes than the class B prototypes, it is classified as class A.
RBF Network Architecture

The above illustration shows the typical architecture of an RBF Network. It consists of an input
vector, a layer of RBF neurons, and an output layer with one node per category or class of data.

Q. Fitness Function in GA?

A. A fitness function is a particular type of objective function that is used to summarise, as a

single figure of merit, how close a given design solution is to achieving the set aims. Fitness functions are
used in genetic programming and genetic algorithms to guide simulations towards optimal design
solutions.

Fitness approximation may be appropriate, especially in the following cases:

 Fitness computation time of a single solution is extremely high

 Precise model for fitness computation is missing
 The fitness function is uncertain or noisy.

Q. Epoch in backpropagation?

A. A training epoch is a complete cycle through your dataset. ... Then you may perform a new forward
pass on the neural network and do another training epoch, looping through your training dataset.
Graphical representation. A really great graphical representation of backpropagation may be found at
this link.
Q. architecture of mcculloch pitts neuron?

A. The McCulloch-Pitts model of a neuron is simple yet has substantial computing potential. It also has a
precise mathematical definition. However, this model is so simplistic that it only generates a binary
output and also the weight and threshold values are fixed.

Q. linearly separable patterns?

A. They are known as linearly separable patterns. Linear separability refers to the fact that classes
of patterns with -dimensional vector can be separated with a single decision surface. ... Conversely, the
two classes must be linearly separable in order for the perceptron network to function correctly [Hay99]

Q. Paradigms of Neural network?

A. Learning, supervised and unsupervised.

Q. Attractors and basin of attractors in Hopfield?

A. An attractor network is a type of recurrent dynamical network, that evolves toward a stable pattern
over time. Nodes in the attractor network converge toward a pattern that may either be fixed-point (a
single state), cyclic (with regularly recurring states), chaotic (locally but not globally unstable) or random
(stochastic).[1] Attractor networks have largely been used in computational neuroscience to model
neuronal processes such as associative memory[2] and motor behavior, as well as in biologically
inspired methods of machine learning. An attractor network contains a set of n nodes, which can be
represented as vectors in a d-dimensional space where n>d. Over time, the network state tends toward
one of a set of predefined states on a d-manifold; these are the attractors.

Basins of attraction. An attractor's basin of attraction is the region of the phase space, over which
iterations are defined, such that any point (any initial condition) in that region will eventually be iterated
into the attractor
Q. stack and surplus variables in lpp?

A. Slack Variables :
Slack variable represents an unused quaintly of resources ; it is added to less than or equal (<) to type
constraints in order to get an equality constraint.
Surplus Variables :
A surplus variable represents the amount by which solution values exceed a resource. These variables
are also called ‘Negative Slack Variables’ . Surplus variables like slack variables carry a zero coefficient in
the objective function. it is added to greater than or equal to (>) type constraints in order to get an
equality constraint.
Artificial Variables :
Artificial variables are added to those constraints with equality (=) and greater than or equal to ( > )
sign. An Artificial variable is added to the constraints to get an initial solution to an LP problem.
Artificial variables have no meaning in a physical sense and are not only used as a tool for
generating an initial solution to an LP problem.