Lecture 6

Radial basis functions neural networks

Neural networks offer a powerful framework for

representing nonlinear mappings from several
inputs to one or more outputs.

There are many different paradigms of neural

networks. Some of them are nonparametric,
e.g. PNN and k-nearest neighbors, and they
do not involve the estimation of a set of pa-
rameters. Some of them are parametric, e.g.
the linear discriminant function.

An important application of neural networks is

regression. Instead of mapping the inputs into
a discrete class label, the neural network maps
the input variables into continuous values. A
major class of neural networks is the radial ba-
sis function (RBF) neural network. We will
look at the architecture of RBF neural net-
works, followed by its applications in both re-
gression and classification.
1
Lecture 6

The architecture of RBF neural network

Denote the input as x, and the output as y(x),

the architecture of a RBF neural network is
given by
M !
X (kx − ci k)2
y(x) = wi exp −
i=1 2σ 2
if we choose to use Gaussian function as basis
functions. Here ci are called centers and σ is
called the width. There are M basis functions
centered at ci. wi are called weights.

c1 w1

wM−1 y
cM−1
+
x
wM
c M

Figure: The architecture of RBF neural net-

work.
2
Lecture 6

Curve fitting using RBF neural networks

The applications of RBF neural networks in

regression can be introduced by examples of
curve fitting. Taking σ = 1, c1 = 2, c2 = 5 and
c3 = 8, the following figures plot the curves by
varying the weights or centers in
3 !
X (x − ci)2
y(x) = wi exp −
i=1 2

The shape of curves are adjustable by chang-

ing the weights or centers. This suggests that
we can use RBF to approximate any unknown
nonlinear function given observational data sam-
ples.

A set of n pairs (x1, t1),..., (xn, tn) is given,

where xi takes real values and ti also takes
values. The RBF neural network is trained
using the data set. The objective is that y(xi)
is as close to ti as possible.
3
Lecture 6

3.5

2.5

2
y

1.5

w2=3 w =2
w1=1.5 3
1

0.5
c1=2 c =5 c =8
2 3

0
0 1 2 3 4 5 6 7 8 9 10
x

3.5

2.5

2
y

1.5

w1=1 w =2 w3=3
2
1

0.5
c1=2 c2=5 c =8
3

0
0 1 2 3 4 5 6 7 8 9 10
x

Figures above; RBF functions for the same

centers and different weights.
4
Lecture 6

3.5

2.5

2
y

1.5

w =1 w2=2 w =3
1 3
1

0.5

c1=2 c =3 c =7
2
3
0
0 1 2 3 4 5 6 7 8 9 10
x

3.5

2.5

2
y

1.5

w1=1 w =2 w =3
2 3
1

0.5
c2=6 c3=8
c =3
1

0
0 1 2 3 4 5 6 7 8 9 10
x

Figures above; RBF functions for same weights

and different centers.
5
Lecture 6

For example, suppose that we have a set of 10

data samples (x1, t1),..., (x10, t10) as given in
the following Table. The data set was gener-
ated by using t = sin(2πx).
i 1 2 3 4 5
xi 0.1 0.2 0.3 0.4 0.5
ti 0.5878 0.9511 0.9511 0.5878 0.0000

i 6 7 8 9 10
xi 0.6 0.7 0.8 0.9 1
ti -0.5878 -0.9511 -0.9511 -0.5878 0.0000

0.8

0.6

0.4

0.2

0
y

−0.2

−0.4

−0.6

−0.8

−1
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
x

Figure: Graphical illustration of the data sam-

ples in above Table.
6
Lecture 6

In general, the training of RBF neural networks

includes the determination of locations of cen-
ters ci, and the estimation of the weights wi.
Note that σ also needs to be set, e.g. as a
constant σ = 1.

We initially concentrate on the estimation of

weights wi, for given centers ci. The meth-
ods on the determination of centers will be
discussed later.

For this example we preset 4 centers as c1 =

0.2, c2 = 0.4, c3 = 0.6 and c4 = 0.8. Here
we also set σ = 1. Thisgives us
 four basis

2 2
functions. exp − (x−0.2)
2 , exp − (x−0.4)
2 ,
   
(x−0.6)2 (x−0.8)2
exp − 2 and exp − 2 .

Over the given ten data samples, form the ma-

trix Φ given by

7
Lecture 6

 
φ φ1,2 φ1,3 φ1,4
 1,1 
 φ2,1 φ2,2 φ2,3 φ3,4 

Φ= ... ... . . . ... 

 
 φ φ9,2 φ9,3 φ9,4 
 9,1 
φ10,1 φ10,2 φ10,3 φ10,4
with
!
(xi − 0.2)2
φi,1 = exp − , i = 1, 2, 3..., 10
2
!
(xi − 0.4)2
φi,2 = exp − , i = 1, 2, 3..., 10
2
!
(xi − 0.6)2
φi,3 = exp − , i = 1, 2, 3..., 10
2
!
(xi − 0.8)2
φi,4 = exp − , i = 1, 2, 3..., 10
2

8
Lecture 6

We can write ten linear equations





φ1,1w1 + φ1,2w2 + φ1,3w3 + φ1,4w4 = t1



 φ2,1w1 + φ2,2w2 + φ2,3w3 + φ2,4w4 = t2

 φ3,1w1 + φ3,2w2 + φ3,3w3 + φ3,4w4 = t3


 ........



 φ10,1w1 + φ10,2w2 + φ10,3w3 + φ10,3w4 = t10



which is
 
φ φ1,2 φ1,3 φ1,4
 1,1
   
 w1 t1
 φ2,1 φ2,2 φ2,3 φ3,4 
 . . .  w2   t2 
 .
. .
. . . . .
. 

 
= ...



 φ  w3   
 9,1 φ 9,2 φ 9,3 φ 9,4  w4 t10
φ10,1 φ10,2 φ10,3 φ10,4 | {z } | {z }
| {z } w t
Φ
or simply
Φw = t
There are 10 equations to solve 4 unknown
parameters (Φ is not square matrix). There is
no exact solution. The least squares estimate
is calculated as
w = (ΦT Φ)−1ΦT t
9
Lecture 6

It is found that
w = [−3083.3, 8903.8, −8892.6, 3071.6]T
The resultant RBF neural network is ready for
prediction for any x
4 !
X (x − ci)2
y(x) = wi exp −
i=1 2
1
RBF model output
RBF model output
0.8 Data samples

0.6

0.4

0.2

0
y

−0.2

−0.4

−0.6

−0.8

−1
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
x

Figure: The results of resultant RBF neural

network for curve fitting. Code sinEX.m have
been used.
10
Lecture 6

A summary of the construction of RBF neural

network for regression.

1. Determine the number and the centers ci;

2. Calculate φi,j for all training data samples;

3. Form matrix Φ and t;

4. Calculate

w = (ΦT Φ)−1ΦT t

5. Use the resultant RBF network

M !
X (kx − cik)2
y(x) = wi exp −
i=1 2σ 2
for prediction on any new sample x.

11