0% found this document useful (0 votes)
7 viewsBack-propagation fuzzy system as nonlinear dynamic system identifiers
This paper for fuzzy systems as controller
Uploaded by
tlemcani_halimCopyright
© © All Rights Reserved
Available Formats
Download as PDF or read online on Scribd
0% found this document useful (0 votes)
7 viewsBack-propagation fuzzy system as nonlinear dynamic system identifiers
This paper for fuzzy systems as controller
Uploaded by
tlemcani_halimCopyright
© © All Rights Reserved
Available Formats
Download as PDF or read online on Scribd
/ 10
BACK-PROPAGATION FUZZY SYSTEM AS NONLINEAR
DYNAMIC SYSTEM IDENTIFIERS
Li-Xin Wang and Jerry M. Mendel
Signal and Image Processing Institute
Department of Electrical Engineering Systems
University of Southern California
Los Angeles, CA 90089-2564
Abstract
Based on the theoretical result that fuzzy systems are universal approximators, we use them as
identifiers for nonlinear dynamic systems. We show that foray systems can be viewed as a three-layer
feedforward network, and develop a back-propsgation algorithm for tesining them to match desired inpat-
output pairs. We present an efficient on-line initial parameter choosing method, which makes the training
gorithm converge very fast, We show a straightforward method to incorporate linguistic [F-THEN rules
bout the nonlinear systems into the tained fuzzy system identifiers so that the latter are determined
‘sing both mumerial and linguistic information in a uniform manner. Finally, we present two examples
(Orbich are the same ae thove used in (I4) to test neural-network Klenifers) which demonstrate how:
(41) the funny system learns to match an unknown nonlinear mapping as training progresses; and, (2)
performance is improved by incorporating linguistic rules,
1 INTRODUCTION
Feedforward neural networks were succesfully used in [14] as identifiers for nonlinear components in dynamic
systems. Theoretical justifeation of this approach is that feedforwaed neural networks ean approximate any
real continuous function on a compact set to arbitrary accuracy {2, 4]. The back-propagation training
algorithm [17, 19, 30] makes it possible to train the neural network identifiers on-line to match unknown
nonlinear mappings.
‘Recently, it was proven that fuzey systems are also capable of approximating any real continuous function
‘on a compact set to arbitrary accuracy (22-24). This result provides a theoretical explanation for the practical
successes of fuzzy systems (1, 5, 6, 9, 13, 20, 21, 31]. This result also motivates us to use fuzzy systems as,
identifiers for nonlinear dynamic systems.
"To use fuszy systems as identifiers for nonlinear dynamic systems, we need to know how to choose their
parameters such that they perform the desired nonlinear mappings. In this paper, we develop a training
algorithm, similar to the back-propagation algorithm for neural networks, to train the fuzzy systems to match
desired input-output pairs, ‘The key ideas in developing this training algorithm are to View a fuzzy system
1s a three-layer feedforward network, and to use the chain rule to determine gradients of the output errors
of the fuzzy system with respect to its design parameters, We show that this training algorithm performs
‘an error back-propagation procedure; hence, we call the fuzzy system equipped with the back-propagation
training algorithm a “Back-Propagation Fuzzy System (BP FS).”
How does the BP FS compate with a back-propagation feedforward neural network (BP FNN) when
‘they ate used as identifiers for nonlinear dynamie systems ? They are similar in that both: (1) are universal
approximators, and therefore qualify as identifiers for nonlinear systems; and, (2) use back-propagation
taining algorithms to adjust their parameters forthe purpose of matching desited input-output pairs
‘There are two important advantages of a BP FS over a BP FNN. First, the parameters of a BP FS
have clear physical meanings; hence (as we show), it is posible to develop a very good method for choosing
its initial parameters. On the other hand, the parameters of a BP FNN have no clear relationships with
0-705 0256292 $3.00 © 19 TERE 1409 input-output data, and therefore theie initial values ate usually chosen randomly. Because both BP traning
algorithms are gradient algorithms, good initial parameters dramatically speeds convergence. ‘The second
advantage ofa BP FS over a BP FNN is that a BP FS provides a natural framework in which to incorporate
human linguistic descriptions (in the form of IF-THEN rules) about the unknown nonlinear system. We
present a method for incorporating linguistic rules (eg, “AND” rules, “OR” rules, ete) into a BP FS.
identifier 90 that it is determined using both numerical and linguistic information in uniform manner,
Frequently, linguistic (subjective) information is dificult to quantify, and is ignored at the frontend of
identifier designs (3, 12]. It is used to help evaluate such designs. Using the results of this paper, the
identifier designer will be able to include subjective information at the front-end of the design, where we
believe it will do the most good.
2 DESCRIPTION AND ANALYSIS OF FUZZY SYSTEMS
In this paper, we consider a fuzzy system whose basic configuration is shown in Fig. 1 (10, 11]. There are four
principal elements in such a fusay system: fuazifcation interface, fuzzy rule base, fuzzy inference machine,
fand defuzaifcation interface. We consider multi-input-single-output fuzzy systems: UC RY — R, where
is compact [16]. A multi-output system can always be separated into a group of single-output systems
‘The fuzsification interface is a mapping from the observed non-fuzzy input space U'C A" to the fuzzy sets
defined in U, where a fussy st [2] defined in U is characterized by a membership function yp : U — (0,1),
and is labelled by a linguistic term F such as “small” very largo”, ete
‘The fuzzy rule base is a cet of linguistic rules in the form of “IF a set of conditions are satisied, THEN a sot
of consequences are inferred.” In this paper, we consider the case that the fuzzy rule base consists of two sets
of IF-THEN rules: one is determined based on desired input-output data pairs using the back-propagation
algorithm (Section 3), which has the form:
Ry IP 2, is A and 22 is AL and .. and zq is AX, THEN = is BM, wo
where j = 1,2, .0M, 24 (7 = 1,2,..n) are the input variables to the furzy system, : isthe output variable of
the fuzzy system, and A and B? age Linguistic etms characterized by fuzzy membership Functions 14:(es)
‘and 4ps(2), respectively; and, the other is linguistic rules provided by human experts, which has the form:
Ri: IF y is Ch ‘LO yp is Ch ‘LO! “LO! tm is Chy
THEN = is D', @
where: yj = 21 for some i= 1,2,..,mand j = 1,2,-,m; api can equal to yj2 for jt # j2; “LO” represents a
logic operation ‘AND’ or ‘OR’; C} and Daze linguistic terms characterized by fuzzy membership functions
ses{ys) and upi(2), respectively: and, = 1,2, 04
"The fuzzy inference machine is decision making logic which employs fuzzy rules from the fuzzy rule base
to determine fuzzy outputs of a furzy system corresponding to its fuzzfied inputs. For detailed discussions
fon different kinds of fuzzy inference machines, se (1!)
‘The defussifcation interface defuzifes the fuzzy outputs ofthe fuzzy inference machine and generates &
non-fuzzy output which is the actual output of the whole furay system. The centroid defuzzification method
[11] is the most commonly-used method, and, we use this method in this paper.
‘There are many different choices within each of the four elements of the fuzzy system, e.g, within the
fuzzy inference machine we may choose “minimum inference", “product inferenes", “arithmetic inference”,
“maximum inference”, “standard sequence inference”, “Boolean inference”, “Goguen's inference”, ete. {11}
For each choice within an element, there are many design parameters, eg. if we choose the membership
functions in the fuzzification interface to be Gaussian as in (4), then the parameters for this choice are af, 2
and of. Consequently, the fuzzy system of Fig. 1 constitutes @ very large function space which maps from
UCR to R, We now consider a subset of this function space
Definition 1: The set of fussy systems with centroid defuszification, denoted by ¥ in the sequel, consists
‘of all functions of the form
fay Pul(a) + Dyn Fags (20)
SS ae )
Fa) = Say + Ee (aa ag(20) where [:0C RY = R,
21,22, 00Fn) € Us pay (2) is the Gaussian membership function defined by
sy l2e) = heen , o
where: of, and of are real-valued parameters with 0 < of < 1; 9 and d! are the points in the output
space at which yp; and pr achieve their maximum values, respectively; w"(z) is the “weight” of Ry in (2)
‘whichis determined by
wiz) = Hoy (y1)"AO" Hey y2)"AO"..."AO" Hcy, (Ym)s ©)
where “AO” isthe algebraic product “*” ifthe corresponding “LO” in (2) is “AND”, and isthe algebraic
sum °£" if the coreesponding “LO” in (2) is “OR”, whore “is defined as 11]: 2ty = 2+ y— 215 and,
dtcs(th) ate membership functions chosen from the set [1] defined next.
Definition 2: Let fy: 4 = 1,2,.-,@} be a finite set of some commonly used membership functions, eg,
‘iy may be triangular, trapezoidal, Gaussian, sigmoid, et., and let the Gaussian membership function (4) be
in this set. The set generated by [uy :q = 1,2,..,Q], denoted as [ig] is defined as all functions of the form
XD oes ©
pez reel.)
where the sum and product operations are over finite p € Z* (Z* is the set of positive integers) and
4 €[1,2,.u,Q], respectively, and any € R. Cleary, 4) includes each py as its element.
Let dial fis fa) = supgev(|fu(a) — fo(z))) be the sup-metric, then (Y,da) is @ meteie space. Using the
Stone-Weierstrass Theorem {13}, we ean prove:
‘THEOREM 1: (Y.du) is dense in (C[U], daa), where C{U] is the set of all real continuous functions
defined on J. In other words, for any given real continuous funetion 9 on the compact set U CR and
abitrary ¢> 0, there exists / € ¥ (Y is defined in Definition 1) such that
supgeulola) = fl < ®
Due to space limitation, we omit the proofs ofall the theorems in this paper. Proofs of similar theorems
ean be found in [22-24]
3 BACK-PROPAGATION TRAINING FOR THE FUZZY SYS-
TEM
Since d! and a in (3) are provided by human experts, our task now is to determine the remaining parameters
P/sMjof- ff and on (2) Fo this purpose, we consider the following fay sub-system in which no linguistic
rules are wed :
Tha PUM yg (20)
Dye 0)
fe
@)
By analysing (8), we observe a very important fact: the mapping f+ U — R determined by (8) can
bbe represented as a three-layer feedforward network, as shown in Fig, 2. Now we consider the following
problem: Given an input-output pair (27, d?), 2? € UCR" and d? € Re, design a fuzzy system fin the form
of (8) such that
Pe hipar)— ary?
@ = e)- # )
is minimized, where by designing f we mean to specify the parameters of f, i, to specify Mf, 3, af,
and of for #= 1.2.00 and j= 12,0, In this pape, we consider the case where M and af ae fied
In fact, choosing AY can be viewed as an order determination problem 3, and, choosing 2, af, #1 and of
Can be viewed as a paramezer estimation problem. We only conser the parameter estimation problem
in this paper. Additionally, we i af = 1 because for practical problems itis reasonable to sssune that
1411 every fuszy membership function achieves unity membership value at some point. We now develop an error
back propagation training algorithm to determine the remaining parameters #8! and of- We wil use ef
and dto denote o”,f(2?) and @, respectively.
‘Totrain 3, we use
Phe t= 24) - ao)
where j= 1,2.0M. = 0.1.2.0, and isa constant stepsine. From Fig. 2 we see that J (and hence ¢)
depends on P only trough o, where f= 9/60 = Fse,p), b= Pty and y= ype ence,
using the ehain rule, we have
ay
ayy
as)
(4)
(18)
where = 2,2,.04m, 9 = Us2y oe Mand b= 0.1.2
‘Using the samme inethed ae above we obain the fllowing taining algorithm for of
okt) = olth-a2
ae f= 21(4))?
= ality a) i)
where §= 1,2,.0)m)J= 1,2, Myand & = 01.2,
“The training gorithm (12), (15) and (16) performe an ertor backpropagation procedure: to train 2,
the “normalized” error (f ~d)/¥ is backepropagated tothe layer of 2, then # ie updated using (12) in which
1p is the input to 2 (see Fig. 2); to train # and of, the “normalized” error (f — d)/6 times (2 — f) and
1 is back-propagated to the processing unit of Layer 1 whose output is y; then, # and o} ate updated
{sing (15) and (16) respectively in which the remaining vatiables #,2f and of (i.e. the variables on the
right-hand sides of (15) and (16), except the back-propagated ertor 4L(s! ~ f)ye) can he obtained locally.
“The taining procedute for the fury system of Fig. 2is «two-pass procedure: fist, fora given input 2P,
compute forward slong the network (ic, the fury system) to obtain y (j= 1,2.-M),a,0 and f; then,
ttain the network parameters 2,22 and of (¢ = 1,2,~-.0,j = 1,2,-4M) backwards using (12), (15) and
(18), respectively:
1412 4 IDENTIFICATION OF NONLINEAR DYNAMIC SYSTEMS
USING THE BACK-PROPAGATION FUZZY SYSTEM
‘Theorem 1 assures us that forty systems with centroid defuztifeation can approximate any nonlinear real
continuous function on the compact set U'C A" to arbitrary accuracy; and, the back propagation training
algorithm developed in the last section gives us a practical way to train the fuzzy system to match desired
input-output pairs. These provide the justification to use back-propagation fuzzy systems as identifiers for
nonlinear dynamic systems. Additionally, the fuzzy systems have two important features which make them
even more attactive as identifers for nonlinear systems.
First, the parameters of the fuzzy systems, 2,3
a] and 2 are points at which the membership functions of the fuzzy sets defined in the input and output
spaces achieve their maximum values, respectively, and, o} characterize the shape of the input membership
functions. Based on these physical meanings, we can develop a very good method for choosing thei inital
values, ax described later inthis section. Because the back-propagation algorithm isa gradient algorithm,
good choice of initial parameters speeds up convergence dramatically.
‘The second attractive featur of furry systems, which may be mote important and essential than the frst
one, is that the fuzzy aystems provide a natural framework to incorporate linguistic descriptions about the
tnknovn nonlinear systems into the identifers Specifically, (3) isthe final fuzzy system identifi in which
@ and w! are determined based on linguistic rales, and the remaining parameters are determined using the
bback-propagation algorithm based on numerical input-output pairs. ‘This feature has practical importance
‘because many real world nonlinear systems are controlled by human experts (eg. aircraft, power systems,
‘economic systems, ec.), and, these experts can provide linguistic descriptions about the nonlinear systems
‘These linguistic descriptions are vague and fuzzy, so that traditional identifiers (3, 12] and neural identifiers
(14,15) cannot make use of them atthe front-end of theie designs. They are used only to help evaluate such
designs. The BP FS provides an idenifer which can make use of both numerical information (in the form of
input-output pairs) and linguistic information (in the form of 1F-THEN rules) at the front-end of its design
in a uniform manner.
‘Based on the discussion in (Id), we use the series-parallel identieation model in which the output of,
the nonlinear plant (rather thas the identification model is fed back into the identification model. Because
wwe only use the seies-paralle identification model, the static BP algorithm developed in the last section is
sufficient to train the identifiers,
‘We now propose the method for on-line initial parameter choosing, and provide a theoretical justification
(heorem 2) for why this is a good method
‘An On-Line Initial Parameter Choosing Method for BP FS: Suppose the nonlinear plant to be identified
tarts operation from F = 0. Do act start the BP training algorithm (12), (15) and (16) forthe frst: Mf
time points. Set the parameters £! (IM) = w(j) and (IM) = g(u(j)), where w(3) = (W(d)o-ms ta) is the
input to both the plant and the identification model, and g(u(s)) isthe desired output of the BP FS for
input u(j); and, set o/(M) equal to some small numbers (see Theotem 2), or set of(M) = [maz(w(j)
1,2) Mf) =min(u({) j= 1,2,.-.4¥)}/2M (this choice makes the input membership functions “uniformly”
cover the range of us(j) from j = 1 to j = M; in all the simulations in Section 5, we use this choice), where
G2 12, Mp4 = 1.2,-.9n. Star the online training for the BP FS identifier from time point M +1,
THEOREM 2 For arbitrary « > 0, there exists @° > O such that the fuzzy system f of (8) with the
above initial # and 2 and o} = o” has the folowing property that
‘and @!, have clear physical meanings. Specifically,
WiCula)) = 1uGaD1 < « a
for all j= 1,2, M
Based on Theorem 2 we see that the initial parameter choosing method is a good one because the fuzzy
system with these initial parameters can at east match the first Mf input-output pairs arbitrarily well. If
these first M input-output pairs contain some important features of the unknown nonlinear mapping, we
may hope that after the back-propagation training starts from time point M + 1, the BP FS identifier will
Converge to the unknown nonlinear mapping very quickly. In fact, based on our simulation results in Section
5, this is indeed true. Hovtever, we cannot choose of to be too samll, because although a fuzzy system with
stall oj matches the fist M pairs quite ell, it will have large approximation errors for other input-output
1413 pits. Therefore, in our simulations, we use the second choice of ef described in the on-line initial parameter
choosing method,
5 SIMULATIONS
We used the same examples as in [14] to simulate our BP FS identifiers, because we want to compare the
BP PS identifiers with neural network identifiers,
Example I: The plant to be identified is govesned by the difference equation
wk +)
O.3y(k) + 0.6y(k 1) + ofute, as)
where the unknown function has the form g(u) = 0.6sin(zu)-+0.3sin(Sru) +0.Lsi
the plant, a series parallel model governed by the difference equation
ru). In order to identity
e+)
Oy A) + 0.6y( = 1) + sulk) ag)
‘was used, where J(e) is of the form (8) with Af = 40. We chose a = 0.5 in the BP training algorithm
(22), (15) and (16), and, we used the on-line initial parameter choosing method in Section 4. We started
the training ftom time point k= 40, and trained che parameters £24 and of for one cyte at each time
point, i, we used (12), (5) and (16) once at each time point (in this case the “E” im (12), (15) and (16)
agrees with the “A” in (18) and (19), Figutes {a)-(d) show the outputs ofthe plant (sli-ine) and the
idemification model (dashesline) when the taining was stopped at & = 190,200,300 and 40, respectively,
‘where the input u(k) = sin(27k/250). Wo see fom Figs. 3(a)}(2) that: 1) the output ofthe identification
‘model fllows the output ofthe plant almost immediately, and sll oes so when the taining was stopped at
2 = 100,200,300 and 400; and, 2) the identieaton madel approximates the plant more and more accurately
as more and more training is perform. fn [the same plant was dented sing a neural network identifier
which failed to follow the plant when the tsining was stopped at & = 600. We see from Fig. J(a) that
our BP FS idemifer follows the plant without large erors even when the training was stopped as eatly a3
= 100, We thnk that the main reason forthe superior performance of the BP FS identifier is that we have
a very good initial parameter choosing method.
we accept the initial parameter choosing method as & good one, what about the BP training algorithm
itself? Can the latter make the identification model converge to the plant when the initial parameters
ate chosen randomly ? Figure 4 shows the outputs of the identification model and the plant for the input
u(h) = sin(2ek/250) for 1 << 250 and S01 < k < TOO and w(E) = O.Ssin(2ak/250) + 0 Ssin(2rt/25)
for 291
