Introduction to Neural Network

Algorithm

Yinghao Wu
Department of Systems and Computational Biology
Albert Einstein College of Medicine
Fall 2014
 Outline

• Background
• Supervised learning (BPNN)
• Unsupervised learning (SOM)
• Implementation in Matlab
 Outline

• Background
• Supervised learning (BPNN)
• Unsupervised learning (SOM)
• Implementation in Matlab
 Biological Inspiration
Idea : To make the computer more robust, intelligent, and learn, …
Let’s model our computer software (and/or hardware) after the brain
 Neurons in the Brain
• Although heterogeneous, at a low level
the brain is composed of neurons
– A neuron receives input from other neurons
(generally thousands) from its synapses
– Inputs are approximately summed
– When the input exceeds a threshold the
neuron sends an electrical spike that travels
that travels from the body, down the axon, to
the next neuron(s)
 Dendrites
Neuron Model
Axon

Cell Body

Synapse

the weight “w” corresponds to the strength of a synapse

the cell body is represented by the summation and the transfer

function

the neuron output “a”represents the signal on the axon

 Supervised Learning

• It is based on a
labeled training set. ε Class

• The class of each

ε Class
A
piece of data in B λ Class

training set is known.

λ Class
B
• Class labels are pre- A
determined and A ε Class

provided in the λ Class B

training phase.
 Unsupervised Learning
• Input : set of patterns P, from n-dimensional space S, but
little/no information about their classification, evaluation,
interesting features, etc.
It must learn these by itself! : )

• Tasks:
– Clustering - Group patterns based on similarity
– Vector Quantization - Fully divide up S into a small set of
regions (defined by codebook vectors) that also helps
cluster P.
– Feature Extraction - Reduce dimensionality of S by
removing unimportant features (i.e. those that do not help
in clustering P)
 Supervised Vs Unsupervised

• Task performed • Task performed

Classification Clustering
Pattern • NN Model :
Recognition
Self Organizing
• NN model : Maps
Preceptron “What groupings exist in
Feed-forward NN this data?”
“How is each data point
“What is the class of this related to the data set
data point?” as a whole?”
 Applications
• Aerospace
– High performance aircraft autopilots, flight path simulations,
aircraft control systems, autopilot enhancements, aircraft
component simulations, aircraft component fault detectors
• Automotive
– Automobile automatic guidance systems, warranty activity
analyzers
• Banking
– Check and other document readers, credit application evaluators
• Defense
– Weapon steering, target tracking, object discrimination, facial
recognition, new kinds of sensors, sonar, radar and image signal
processing including data compression, feature extraction and
noise suppression, signal/image identification
• Electronics
– Code sequence prediction, integrated circuit chip layout, process
control, chip failure analysis, machine vision, voice synthesis,
nonlinear modeling
 Applications
• Financial
– Real estate appraisal, loan advisor, mortgage screening, corporate
bond rating, credit line use analysis, portfolio trading program,
corporate financial analysis, currency price prediction
• Manufacturing
– Manufacturing process control, product design and analysis,
process and machine diagnosis, real-time particle identification,
visual quality inspection systems, beer testing, welding quality
analysis, paper quality prediction, computer chip quality analysis,
analysis of grinding operations, chemical product design analysis,
machine maintenance analysis, project bidding, planning and
management, dynamic modeling of chemical process systems
• Medical
– Breast cancer cell analysis, EEG and ECG analysis, prosthesis
design, optimization of transplant times, hospital expense
reduction, hospital quality improvement, emergency room test
advisement
 Outline

• Background
• Supervised learning (BPNN)
• Unsupervised learning (SOM)
• Implementation in Matlab
 Neural Networks
● Artificial neural network (ANN) is a machine learning
approach that models human brain and consists of a number
of artificial neurons.
● Neuron in ANNs tend to have fewer connections than
biological neurons.
● Each neuron in ANN receives a number of inputs.
● An activation function is applied to these inputs which results
in activation level of neuron (output value of the neuron).
● Knowledge about the learning task is given in the form of
examples called training examples.
 Contd..
● An Artificial Neural Network is specified by:
− neuron model: the information processing unit of the NN,
− an architecture: a set of neurons and links connecting neurons.
Each link has a weight,
− a learning algorithm: used for training the NN by modifying the
weights in order to model a particular learning task correctly on
the training examples.
● The aim is to obtain a NN that is trained and generalizes
well.
● It should behaves correctly on new instances of the
learning task.
 Neuron
● The neuron is the basic information processing unit of a
NN. It consists of:
1 A set of links, describing the neuron inputs, with weights W1, W2,
…, Wm
2 An adder function (linear combiner) for computing the weighted
sum of the inputs: m
(real numbers) u = ∑ w jx j
j=1

3 Activation function ϕ for limiting the amplitude of the neuron

output. Here ‘b’ denotes bias.

y = ϕ (u + b)
 The Neuron Diagram
Bias
b
x1 w1
Activation
Induced function
Field
Output

Input
x2 w2
∑ v
ϕ(−) y
values

Summing
function

xm wm
weights
 Neuron Models
● The choice of activation function ϕ determines the
neuron model.
Examples:
 a if v < c
● step function: ϕ ( v ) = 
 b if v > c

 a if v < c
● ramp function: 
ϕ ( v ) =  b if v > d
 a + (( v − c )( b − a ) /( d − c )) otherwise


● sigmoid function with z,x,y parameters

1
ϕ (v ) = z +
1 + exp( − xv + y )
● Gaussian function: 1  1  v − µ 2 
ϕ (v ) = exp  −   

2π σ  2 σ  
Step Function

c
Ramp Function

c d
Sigmoid function
 Network Architectures
● Three different classes of network architectures

− single-layer feed-forward
− multi-layer feed-forward
− recurrent

● The architecture of a neural network is linked

with the learning algorithm used to train
 Single Layer Feed-forward

Input layer Output layer

of of
source nodes neurons
 Perceptron: Neuron Model
(Special form of single layer feed forward)

− The perceptron was first proposed by Rosenblatt (1958) is a simple

neuron that is used to classify its input into one of two categories.
− A perceptron uses a step function that returns +1 if weighted sum
of its input ≥ 0 and -1 otherwise

 + 1 if v ≥ 0
ϕ ( v ) = 
 − 1 if v < 0
b (bias)
x1
w1
v y
x2 w2
ϕ(v)
wn
xn
 Perceptron for Classification
● The perceptron is used for binary classification.
● First train a perceptron for a classification task.
− Find suitable weights in such a way that the training examples are
correctly classified.
− Geometrically try to find a hyper-plane that separates the examples of the
two classes.
● The perceptron can only model linearly separable classes.
● When the two classes are not linearly separable, it may be
desirable to obtain a linear separator that minimizes the mean
squared error.
● Given training examples of classes C1, C2 train the perceptron in
such a way that :
− If the output of the perceptron is +1 then the input is assigned to class C1
− If the output is -1 then the input is assigned to C2
 Learning Process for Perceptron

● Initially assign random weights to inputs between -0.5 and

+0.5
● Training data is presented to perceptron and its output is
observed.
● If output is incorrect, the weights are adjusted accordingly
using following formula.
wi ← wi + (a* xi *e), where ‘e’ is error produced
and ‘a’ (-1 < a < 1) is learning rate
− ‘a’ is defined as 0 if output is correct, it is +ve, if output is too low and
–ve, if output is too high.
− Once the modification to weights has taken place, the next piece of
training data is used in the same way.
− Once all the training data have been applied, the process starts again
until all the weights are correct and all errors are zero.
− Each iteration of this process is known as an epoch.
 Example: Perceptron to learn OR
function
● Initially consider w1 = -0.2 and w2 = 0.4
● Training data say, x1 = 0 and x2 = 0, output is 0.
● Compute y = Step(w1*x1 + w2*x2) = 0. Output is correct so
weights are not changed.
● For training data x1=0 and x2 = 1, output is 1
● Compute y = Step(w1*x1 + w2*x2) = 0.4 = 1. Output is correct
so weights are not changed.
● Next training data x1=1 and x2 = 0 and output is 1
● Compute y = Step(w1*x1 + w2*x2) = - 0.2 = 0. Output is
incorrect, hence weights are to be changed.
● Assume a = 0.2 and error e=1
wi = wi + (a * xi * e) gives w1 = 0 and w2 =0.4
● With these weights, test the remaining test data.
● Repeat the process till we get stable result.
Boolean function OR – Linearly separable

X1

1 true true

false true
0 1 X2
 Perceptron: Limitations
● The perceptron can only model linearly separable
functions,
− those functions which can be drawn in 2-dim graph and single
straight line separates values in two part.
● Boolean functions given below are linearly separable:
− AND
− OR
− COMPLEMENT
● It cannot model XOR function as it is non linearly
separable.
− When the two classes are not linearly separable, it may be
desirable to obtain a linear separator that minimizes the mean
squared error.
 XOR – Non linearly separable function

● A typical example of non-linearly separable function is the

XOR that computes the logical exclusive or..
● This function takes two input arguments with values in {0,1}
and returns one output in {0,1},
● Here 0 and 1 are encoding of the truth values false and
true,
● The output is true if and only if the two inputs have
different truth values.
● XOR is non linearly separable function which can not be
modeled by perceptron.
● For such functions we have to use multi layer feed-forward
network.
 Input Output
X1 X2 X1 XOR X2
0 0 0
0 1 1
1 0 1
1 1 0

These two classes (true and false) cannot be separated using a

line. Hence XOR is non linearly separable.

X1

1 true false

false true
0 1 X2
 Multi layer feed-forward NN (FFNN)

● FFNN is a more general network architecture, where there are

hidden layers between input and output layers.
● Hidden nodes do not directly receive inputs nor send outputs to
the external environment.
● FFNNs overcome the limitation of single-layer NN.
● They can handle non-linearly separable learning tasks.

Input Output
layer layer

Hidden Layer
3-4-2 Network
 FFNN for XOR

● The ANN for XOR has two hidden nodes that realizes this non-linear
separation and uses the sign (step) activation function.
● Arrows from input nodes to two hidden nodes indicate the directions of
the weight vectors (1,-1) and (-1,1).
● The output node is used to combine the outputs of the two hidden
nodes.
Input nodes Hidden layer Output layer Output

H1 –0.5
X1 1
–1 1
Y
–1 H2
X2 1 1
 Inputs Output of Hidden Nodes Output X1 XOR X2
X1 X2 H1 H2 Node
0 0 0 0 –0.5
0 0
0 1 –1
0 1 0.5
1 1
1 0 1 –1
0 0.5
1 1
1 1 0 0 –0.5
0 0
Since we are representing two states by 0 (false) and 1 (true), we
will map negative outputs (–1, –0.5) of hidden and output layers
to 0 and positive output (0.5) to 1.
 +1

1
3
2

+1
 Three-layer networks
x1

x2

Input Output

xn

Hidden layers
 What do each of the layers do?

3rd layer can generate

1st layer draws 2nd layer combines arbitrarily complex
linear boundaries the boundaries boundaries
Properties of architecture

• No connections within a layer

Each unit is a perceptron

m
y i = f (∑ w ij x j + bi )
j= 1
Properties of architecture

• No connections within a layer

• No direct connections between input and output layers
•

Each unit is a perceptron

m
y i = f (∑ w ij x j + bi )
j= 1
Properties of architecture

• No connections within a layer

• No direct connections between input and output layers
• Fully connected between layers
•

Each unit is a perceptron

m
y i = f (∑ w ij x j + bi )
j= 1

39
Properties of architecture

• No connections within a layer

• No direct connections between input and output layers
• Fully connected between layers
• Often more than 3 layers
• Number of output units need not equal number of input units
• Number of hidden units per layer can be more or less than
input or output units

Each unit is a perceptron

m
y i
= f (∑ w ij
x j
+ bi )
j= 1

Often include bias as an extra weight

Backpropagation learning algorithm ‘BP’

Solution to credit assignment problem in MLP. Rumelhart, Hinton and

Williams (1986) (though actually invented earlier in a PhD thesis
relating to economics)

BP has two phases:

Forward pass phase: computes ‘functional signal’, feed forward

propagation of input pattern signals through network

Backward pass phase: computes ‘error signal’, propagates

the error backwards through network starting at output units
(where the error is the difference between actual and desired
output values)
Conceptually: Forward Activity -
Backward Error

42
 Forward Propagation of Activity
• Step 1: Initialize weights at random, choose a
learning rate η
• Until network is trained:
• For each training example i.e. input pattern and
target output(s):
• Step 2: Do forward pass through net (with fixed
weights) to produce output(s)
– i.e., in Forward Direction, layer by layer:
• Inputs applied
• Multiplied by weights
• Summed
• ‘Squashed’ by sigmoid activation function
• Output passed to each neuron in next layer
– Repeat above until network output(s) produced
Step 3. Back-propagation of error
• Compute error (delta or local gradient) for each
output unit δ k
• Layer-by-layer, compute error (delta or local
gradient) for each hidden unit δ j by backpropagating
errors

Step 4: Next, update all the weights ∆wij

By gradient descent, and go back to Step 2
− The overall MLP learning algorithm, involving
forward pass and backpropagation of error
(until the network training completion), is
known as the Generalised Delta Rule (GDR),
or more commonly, the Back Propagation
(BP) algorithm
 Learning Algorithm:
Backpropagation
The following slides describes teaching process of multi-layer neural network
employing backpropagation algorithm. To illustrate this process the three layer neural
network with two inputs and one output,which is shown in the picture below, is used:
 Learning Algorithm:
Backpropagation
Each neuron is composed of two units. First unit adds products of weights coefficients and
input signals. The second unit realise nonlinear function, called neuron transfer (activation)
function. Signal e is adder output signal, and y = f(e) is output signal of nonlinear element.
Signal y is also output signal of neuron.
 Learning Algorithm:
Backpropagation
Pictures below illustrate how signal is propagating through the network,
Symbols w(xm)n represent weights of connections between network input xm and
neuron n in input layer. Symbols yn represents output signal of neuron n.
Learning Algorithm:
Backpropagation
Learning Algorithm:
Backpropagation
 Learning Algorithm:
Backpropagation
Propagation of signals through the hidden layer. Symbols wmn represent weights
of connections between output of neuron m and input of neuron n in the next
layer.
Learning Algorithm:
Backpropagation
Learning Algorithm:
Backpropagation
 Learning Algorithm:
Backpropagation
Propagation of signals through the output layer.
 Learning Algorithm:
Backpropagation
In the next algorithm step the output signal of the network y is
compared with the desired output value (the target), which is found in
training data set. The difference is called error signal d of output layer
neuron
 Learning Algorithm:
Backpropagation
The idea is to propagate error signal d (computed in single teaching step)
back to all neurons, which output signals were input for discussed
neuron.
 Learning Algorithm:
Backpropagation
The idea is to propagate error signal d (computed in single teaching step)
back to all neurons, which output signals were input for discussed
neuron.
 Learning Algorithm:
Backpropagation
The weights' coefficients wmn used to propagate errors back are equal to
this used during computing output value. Only the direction of data flow
is changed (signals are propagated from output to inputs one after the
other). This technique is used for all network layers. If propagated errors
came from few neurons they are added. The illustration is below:
 Learning Algorithm:
Backpropagation
When the error signal for each neuron is computed, the weights
coefficients of each neuron input node may be modified. In formulas
below df(e)/de represents derivative of neuron activation function
(which weights are modified).
 Learning Algorithm:
Backpropagation
When the error signal for each neuron is computed, the weights
coefficients of each neuron input node may be modified. In formulas
below df(e)/de represents derivative of neuron activation function
(which weights are modified).
 Learning Algorithm:
Backpropagation
When the error signal for each neuron is computed, the weights
coefficients of each neuron input node may be modified. In formulas
below df(e)/de represents derivative of neuron activation function
(which weights are modified).
MLP/BP: A worked example
Worked example: Forward Pass
Worked example: Forward Pass
Worked example: Backward Pass
Worked example: Update Weights
Using Generalized Delta Rule (BP)
Similarly for the all weights wij:
Verification that it works
 Training
• This was a single iteration of back-prop
• Training requires many iterations with many
training examples or epochs (one epoch is entire
presentation of complete training set)
• It can be slow !
• Note that computation in MLP is local (with
respect to each neuron)
• Parallel computation implementation is also
possible
 Training and testing data
• How many examples ?
– The more the merrier !
• Disjoint training and testing data sets
– learn from training data but evaluate
performance (generalization ability) on
unseen test data
• Aim: minimize error on test data
 Outline

• Background
• Supervised learning (BPNN)
• Unsupervised learning (SOM)
• Implementation in Matlab
 Unsupervised Learning – Self Organizing
Maps
• Self-organizing maps (SOMs) are a data visualization
technique invented by Professor Teuvo Kohonen
– Also called Kohonen Networks, Competitive Learning,
Winner-Take-All Learning
– Generally reduces the dimensions of data through the use
of self-organizing neural networks
– Useful for data visualization; humans cannot visualize high
dimensional data so this is often a useful technique to
make sense of large data sets
 Basic “Winner Take All” Network
• Two layer network
– Input units, output units, each input unit is connected to each
output unit
Input Layer Output Layer

I1
O1

I2

O2
I3
Wi,j
 Typical Usage: 2D Feature Map
• In typical usage the output nodes form a 2D “map” organized
in a grid-like fashion and we update weights in a
neighborhood around the winner

Output Layers
Input Layer
O11 O12 O13 O14 O15
I1
O21 O22 O23 O24 O25

I2 O31 O32 O33 O34 O35

… O41 O42 O43 O44 O45

I3
O51 O52 O53 O54 O55
 Basic Algorithm
– Initialize Map (randomly assign weights)
– Loop over training examples
• Assign input unit values according to the values in the current
example
• Find the “winner”, i.e. the output unit that most closely
matches the input units, using some distance metric, e.g.

For all output units j=1 to m n 2

and input units i=1 to n
Find the one that minimizes:
∑ (W
i =1
ij − Ii )
• Modify weights on the winner to more closely match the
input
∆W t +1 = c( X it − W t )
where c is a small positive learning constant
that usually decreases as the learning proceeds
 Result of Algorithm
• Initially, some output nodes will randomly be a little
closer to some particular type of input
• These nodes become “winners” and the weights
move them even closer to the inputs
• Over time nodes in the output become
representative prototypes for examples in the input
• Note there is no supervised training here
• Classification:
– Given new input, the class is the output node that is the
winner
 Modified Algorithm
– Initialize Map (randomly assign weights)
– Loop over training examples
• Assign input unit values according to the values in the current
example
• Find the “winner”, i.e. the output unit that most closely matches
the input units, using some distance metric, e.g.
• Modify weights on the winner to more closely match the input
• Modify weights in a neighborhood around the winner so the
neighbors on the 2D map also become closer to the input
– Over time this will tend to cluster similar items closer on the map
 Updating the Neighborhood
• Node O44 is the winner
– Color indicates scaling to update neighbors

Output Layers
∆W t +1 = c( X it − W t )
O11 O12 O13 O14 O15

O21 O22 O23 O24 O25

c=1 Consider if O42 is
winner for some
O31 O32 O33 O34 O35 c=0.75 other input; “fight”
over claiming O43,
O41 O42 O43 O44 O45 O33, O53
c=0.5
O51 O52 O53 O54 O55
 Selecting the Neighborhood
• Typically, a “Sombrero Function” or Gaussian
function is used

• Neighborhood size usually decreases over time to

allow initial “jockeying for position” and then “fine-
tuning” as algorithm proceeds
 Outline

• Background
• Supervised learning (BPNN)
• Unsupervised learning (SOM)
• Implementation in Matlab
 Implementation

1. Loading data source.

2. Selecting attributes required.
3. Decide training, validation, and testing data.
4. Data manipulations and Target generation.
(for supervised learning)
5. Neural Network creation (selection of network
architecture) and initialisation.
6. Network Training and Testing.
7. Performance evaluation.
 Loading and Saving data
• load: retrieve data from disk.
– In ascii or .mat format.

>> data = load(‘nndata.txt’);

>> whos data;
Name Size Bytes Class
data 826x7 46256 double array

• Save: saves variables in matlab environment in .mat

format.
>> save nnoutput.txt x, y ,z;
 Matrix manipulation
• region = data(:,1);
• training = data([1:500],:)
• w=[1;2]; w*w’ => [1,2;2,4];
1 2
2 4

• w=[1,2;2,4]; w.*w => [1,4;4,16];

1 4
4 16
 Plotting Data
• plot : plot the vector in 2D or 3D
>> y = [1 2 3 4]; figure(1); plot(power(y,2));

Redefine x axis:
>> x = [2 4 6 8];
>> plot(x,power(y,2));
 Network creation
>>net = newff(PR,[S1 S2...SNl],{TF1 TF2...TFNl},BTF,BLF,PF)

• PR - Rx2 matrix of min and max values for R input elements.

• Si - Size of ith layer, for Nl layers.
• TFi - Transfer function of ith layer, default = 'tansig'.
• BTF - Backprop network training function,
• default = 'trainlm'.
• BLF - Backprop weight/bias learning function,
• default = 'learngdm'.
• PF - Performance function,
• default = 'mse’
• newff : create and returns “net” = a feed-forward backpropagation network.
 Network creation (cont.)
S2: number of
ouput neuron

S1: number
hidden neurons

Number of inputs
decided by PR
 Network Initialisation
-1 1 neuron 1
>> PR = [-1 1; -1 1; -1 1; -1 1];
-1 1
-1 1
Min Max
-1 1

• Initialise the net’s weighting and biases

• >> net = init(net); % init is called after newff

• re-initialise with other function:

– net.layers{1}.initFcn = 'initwb';
– net.inputWeights{1,1}.initFcn = 'rands';
– net.biases{1,1}.initFcn = 'rands';
– net.biases{2,1}.initFcn = 'rands';
 Neurons activation
>> net = newff([-1 1; -1 1; -1 1; -1 1], [4,1], {‘logsig’ ‘logsig’});

TF2: logsig

TF1: logsig
Network Training

• The overall architecture of your neural network is store in

the variable net;
• variable can be reset.

net.trainParam.epochs =1000; (Max no. of epochs to train) [100]

net.trainParam.goal =0.01; (stop training if the error goal hit) [0]
net.trainParam.lr =0.001; (learning rate, not default trainlm) [0.01]
net.trainParam.show =1; (no. epochs between showing error) [25]
net.trainParam.time =1000; (Max time to train in sec) [inf]
 net.trainParam parameters:

• epochs: 100
• goal: 0
• max_fail: 5
• mem_reduc: 1
• min_grad: 1.0000e-010
• mu: 0.0010
• mu_dec: 0.1000
• mu_inc: 10
• mu_max: 1.0000e+010
• show: 25
• time: Inf
 net.trainFcn options
• net.trainFcn=trainlm ; a variant of BP based on second
order algorithm (Levenberg-Marquardt)
 Network Training(cont.)
TRAIN trains a network NET according to NET.trainFcn and NET.trainParam.

>> TRAIN(NET,P,T,Pi,Ai)
• NET - Network.
• P - Network inputs.
• T - Network targets, default = zeros.
• Pi - Initial input delay conditions, default = zeros.
• Ai - Initial layer delay conditions, default = zeros.

>> p = [-0.5 1 -0.5 1; -1 0.5 -1 0.5; 0.5 1 0.5 1; -0.5 -1 -0.5 -1];

-0.5 1 -0.5 1
-1 0.5 -1 0.5
0.5 1 0.5 1
For
Training -0.5 -1 -0.5 -1 neuron 1
pattern 1
 Network Training(cont.)
>>TRAIN(NET,P,T,Pi,Ai)
• NET - Network.
• P - Network inputs.
• T - Network targets, default = zeros. (optional only for NN with targets)
• Pi - Initial input delay conditions, default = zeros.
• Ai - Initial layer delay conditions, default = zeros.

>> p = [-0.5 1 -0.5 1; -1 0.5 -1 0.5; 0.5 1 0.5 1; -0.5 -1 -0.5 -1];

>> t = [-1 1 -1 1];

>> net = train(net, p, t);

-1 1 -1 1

Training
pattern 1
 Simulation of the network
>> [Y] = SIM(model, UT)
• Y : Returned output in matrix or structure format.
• model : Name of a block diagram model.
• UT : For table inputs, the input to the model is interpolated.

>> UT = [-0.5 1 ; -0.25 1; -1 0.25 ; -1 0.5];

>> Y = sim(net,UT);

-0.5 1.00
-0.25 1.00
-1.00 0.25
For
Training -1.00 0.50 neuron 1
pattern 1
 Performance Evaluation

• Comparison between target and network’s

output in testing set.
• Comparison between target and network’s
output in training set.
• Design a metric to measure the
distance/similarity of the target and output,
or simply use mse.
 NEWSOM
• Create a self-organizing map.
>> net = newsom(PR,[d1,d2,...],tfcn,dfcn,olr,osteps,tlr,tns)

• PR - Rx2 matrix of min and max values for R input elements.

• Di - Size of ith layer dimension, defaults = [5 8].
• TFCN - Topology function, default = 'hextop'.
• DFCN - Distance function, default = 'linkdist'.
• OLR - Ordering phase learning rate, default = 0.9.
• OSTEPS - Ordering phase steps, default = 1000.
• TLR - Tuning phase learning rate, default = 0.02;
• TND - Tuning phase neighborhood distance, default = 1.
 NewSom parameters

• The topology function TFCN can be HEXTOP, GRIDTOP, or

RANDTOP.
• The distance function can be LINKDIST, DIST, or MANDIST.

• Exmple:
>> P = [rand(1,400)*2; rand(1,400)];
>> net = newsom([0 2; 0 1],[3 5]);
>> plotsom(net.layers{1}.positions)

TRAINWB1 By-weight-&-bias 1-vector-at-a-time training function

>> [net,tr] = trainwb1(net,Pd,Tl,Ai,Q,TS,VV,TV)
 Outline

• Background
• Supervised learning (BPNN)
• Unsupervised learning (SOM)
• Implementation in Matlab
• Applications