Neural Networks and Fuzzy Logic in Intelligent Control

Hainid Bereiiji
Sterling Federal S y s t e m
Artificial Intelligeiice Research Branch, Mail Stop: 244-17
NASA Ames Research Center, Moffett Field, CA 94035
E-mail: bereiiji~pluto.arc.casa.gov

Abstract
This p a p a provides an introduction to fuzzy con-
trollers and neural network controllers and dis-
cusses the methods under study to merge tlicir
capabilities for designing hybrid neuro-fuzzy con-
trollers (NFCs). NFCs provide the knowledge
representation power of fuzzy controllers and the
Input Hldden smut
learning capabilities of artificial neural networks. layer layer layer
Several examples are discussed to contrast the ar-
chitecture of the NFCs with individual. fuzzy con-
trollers or neural network controllers.

(b) l o r -
Introduction
The recent growth in attention to ncural networks on /

0
one hand and fuzzy logic on the other, has led to nmiy
suggestions for tbzir coiizbined use in control. Many Figure 1: (a)- General Back-propagation network
simulation programs and actual hardware iiiiplcmenta- (b)- A sigiiioidal activation function
tions of systems are now available which deinonstrate
that these techniques can actually work in applied do-
mains. General back-propagation method
This paper provides perspectives on application of 111 amulti-layer neural network of feedforward type, in-
neural networks and fuzzy logic in intelligent control. put nodes record the features and pass activation val-
It consists of t h e e major sections. In section one. we ues to the output layer through a hidden layer. The
will review the major elements of ncurocontrol, a teiiii addition of the hidden layers to the two-layer percep
we use to refer to tlie neural networks which serve as tron networks allows these networks to represent any
controllers. Special concentration will be on the learn- continuous mapping from input to output. The Back-
ing behavior of these networks. propagation method of Rumelhart et. al. [22] adjusts
In the second section, we will discuss the basic ar- the connection weights of the network so as to improve
chitecture of the fuzzy control technique. the match between the output of the network and the
The third section of this paper is devoted to recent correct results. The addition of the hidden units makes
research on integrating neural networks with the fuzzy learning in these networks more difficult but allows
logic control. It has been shown that both of these much more flexibility in the types of problem that
techniques can use interpolative reasoning which en- these networks can solve.
ables them to go beyond the traditional True-False re-
striction of the AI symbolic methods. Adaptive Heuristic Critic
Neural networks have been studied in the context of
Neural Network Controllers the credit assignment problem by Barto, Sutton, and
Neural network models are increasingly being applied Anderson [1,3] who developed the Adaptive Heuristic
in control [2]. Here we concentrate on the two which Critic method (ARC). Credit assignment in rule-based
have been shown to be good candidates for use in con- systems is the probleiii of how to reinforce individual
trol. rules in a chain when an external reward is given only

916
TH0333-5/90/0000/0916$01.OO 0 1990 IEEE
after the conclusion of the chain. The AHC algoritlim control rules a t the same time, a confiict resolution
learns incrementally during a trial and need not wait strategy is needed. A Maz-Min compositional rule of
until the end of a trial. This feature of AHC algorithm inference, as explained below, is commonly used.
way later formalized by Sutton in the Teinporal Dif- Assume that we have the following two rules:
ference (TD) inethod [24]. Lee [lG] and Lee SC Berenji Rule 1: I F X is A1 and Y is B1 THEN Z is C1
[18] have applied this nietliod to the fuzzy control of a Rule 2: IF X is A2 and Y is B2 THEN Z is C2
cart pole balancing system.
Anderson has extended ABC to tlie case of multi- Now, if we have 21 and y1 as the sensor readings for
layer networks [l].This system uses Rumeliiart’s back- fuzzy variables X and Y ,then their irzlih values are
propagation method [22] and is more flexible than the represented by PA, (21) and p~~( y l ) respectively for
AHC algorithm but learns much more slowly. Berenji Rule 1, where LA^ represents the membership function
[5] has presented an architecture for using this system for A l . Similarly for Rule 2, we have ~ ~ ~ ( 2and 1 )
in fuzzy control. po,(y1) as the truth values of the preconditions. Then
tlie strength of Rule 1 can be calculated by:
Fuzzy Logic Based Controllers a1 = pAl(Z1) A pB1(Y1)-
A difficulty in employing AI techniques in real-time Similarly for Rule 2:
control is how to handle imprecision in tlie knowledge
expressed by human expert operators. Fuzzy set tlie- Q2 = PAa(21) A pB2(Y1)-
ory provides a facility to express the iinprecise knowl-
The effect of the strength of Rule 1 on its conclusion
edge by using linguistic variables [29]. The basic idea
is calculated by:
in fuzzy control centers around the labeling process, in
which the reading of a sensor is translated into a label Pc; (U) = a1 A PCI (U),
as done by human operators. For example, in the con-
text of controlling a nuclear reactor [9], an observed and for Rule 2:
reactor period (i.e., the rate of rise of the power) might
be classified as too short, short, or negative. It is iin- pc;(w) = a2 A pc2(w).
portant to note that the transition between the labels This means that as a result of reading sensor values
are continuous rather than abrupt. This nieans that 21 and yl, Rule 1 is recommending a control action
a reactor’s period of 90 seconds might be termed too ) its niembership function and Rule 2
with p ~ ; ( w as
short to degree 0.2, short to degree 1.0, and negative is recommending a control action with p c ; ( w ) as its
to degree 0.0 [9]. A similar concept was used in our membership function. The conflict-resolution process
cart-pole balancing experiment [SI where an angular
then produces
position of say 5 degrees might be called Positive to
a degree of .8 and Zero (i.e., a label used to describe PC@) = pc; (w)wc:(w) = [alAPcl ( w ) l v ~ ~ Z ~ ~ c , ( w ) l
very small angles) to degree of 0.2. This idea of par-
tial matching plays an important role in fuzzy control, where p ~ ( w is) a pointwise membership function for
and is related to the concept of a membership function the combined conclusion of Rule 1 and Rule 2. The A
used in fuzzy set theory where the boundary of a set is and v operators in above are defined to be the min and
not sharp and the degree of membership specifies how max functions respectively [19]. The result of this last
strongly an clement belongs to a set. operation ( p c ( w ) ) has to be translated (defuzzified) to
The knowledge base of an approxiniate reasoning- a single value. This necessary operation produces a
based controller is a collection of linguistic control rules nonfuzzy control action that best represents the mem-
wliicli are described using linguisiic variables[29]. For bership function of an inferred fuzzy control action.
example, The Center Of Area (COA) method (see [17]) can be
used here. Assuming a discrete universe, we have
IF X is A and Y is B THEN Z is C
is a linguistic control rule where X and Y are sen-
sor readings from the plant and 2 corresponds to the
output (i.e., the recommended action). A , B , and C where n is the number of quantization levels of the
are linguistic values such as LARGE, POSITIVE, etc. output.
which are represented by membership functions (usu-
ally in triangular or trapezoidal forms). When partic- Hybrid NeuroFuzzy Controllers
ular values for X and Y are sensed, then these val- Similarities exist between the neural networks and the
ues are matched against the membership functions of fuzzy logic controllers. They both can handle extreme
A and B respectively. As a result of this matching, nonlinearities in the system. Both techniques allow in-
the degree that each precondition is satisfied will be terpolative reasoning which frees us from the truelfalse
known. Since sensor readings usually trigger several restriction of logical systems such as the ones used in
 Evalualion
symbolic AI. For example, once a iieural network l i ~ Ne1 wor k
been trained for a set of data, it can interpolate and
produce answers for the cases not present in the train-
fallure Signal
ing data set. Similar properties hold for a fuzzy con-
troller. The weighted average sclieme of fuzzy control
and the sum of the products of the ncural nets arc
similar in principle.
The main idea in integrating the fuzzy logic con-
trollers with neural networks is to use tlie streiigtli of
each one collectively in the resulting ncuro-fuzzy con-
trol system. This fusion allows:
1. A human understandable expression of the knowl-
edge used in control in terms of the fuzzy control
rules. This rcduces the difficulties in describing the
trained neural network which is usually treated as a
black box.
2. The fuzzy controller learns to adjust its pcrforinance
automatically using a neural network structure and
hence learns by accumulating experience. I I ,

The main emphasis in this research so far has bccn Figure 2: The NFC Architecture
on automatic design and fine-tuning of the member-
ship functions used in fuzzy control through learning
by neural networks. Here, we concentrate on only thrce
hybrid models but many more references are available on tlie back-propagation algorithm. Figure 2 presents
(such as the proceedings of Iizuka-SS and Iizuka-00 con- tlie architecture of the proposed NFC model. The a p
ferences [27], [GI). plication of this model to the rendezvous and docking
operation of the Space Shuttle is currently being ex-
Fuzzy Control and AHC plored [15].
Lee and Berenji [lS] and Lee[lG] have combined tlie
Adaptive Heuristic Critic model of Barto, Sutton, and Use of Clustering
Aiiderson with fuzzy control to design the m c m t c r -
ship functions of tlie conclusions of tlic control rules. Takagi and Hayashi 1251 have presented an algorithm
This work builds on Barto e t . ups pioneering work which consists of three parts. The first part finds out a
on applying neural networks in control. Two neural- suitable partition of the training data by using a clus-
like elements are used in this model. The Adaptive tering algorithm. Once the best partition of the data is
Heuristic Critic learns by updating tlie predictions of found, then tlie second part of the algorithm identifies
the system’s failure over time. The integrated fuzzy- tlie the mcnibership functions of the IF parts of the
AHC model has been tested in the doniain of cart-pole rules. The last step of the algorithm is to determine
balancing and the results have been consistently bet- the amount of control for each rule. A neural net-
ter when compared with the perforinance of the AHC work is used for the identification of the menibership
model alone (e.g., in terins of speed of leariling and functions in the second step above. After this network
smoothness of control). However, this model is dif- is trained, it will assign the correct rule number to
ficult to apply for other control systems mainly due the combination of different sensor readings. To iden-
to the fact that the developn~entof tlie mathematical tify tlie THEN part of a rule, a backward elimination
functions for the trace function and credit assigninclit method is used. This method arbitrarily eliminates an
are not trivial. The structure proposed here suffers input variable and the neural network of each THEN
from the lack of generality and it may be difficult for part is retrained. This process determines the input
application to larger scale systems. variable with miniinal effects.
Takagi and Hayashi’s model is very sinlilar to
Fuzzy Control and two layer networks Berenji’s work in inductive learning and fuzzy control
Berenji (51 has proposed a Neuro-Fuzzy Controller [6] where an AI clustering method (C4, a descendent of
(NFC) architecture which extends the Fuzzy-AHC Quinlan’s ID3 algorithm) is used. In Berenji’s model,
model mentioned in the last section. In NFC, two a decision theoretic measure is used to decide upon
networks replace the two neural-like elements. These which input variable the decision tree should use to
networks are referred to as the evaluation network and branch on next. The number of leaves of the resulting
the action network. Action networks and evaluation decision tree will indicate the number of control rules
networks are multi-layered networks which are based needed.

918
Perspectives on Fuzzy-" models [4] Bavarian, B., Introduction to Neural Networks for
Due to the space limitation, several otlicr major ap- Intelligent Control, IEEE Control Systems Mag&
proaches to using a hybrid neural networks and fuzzy zinc, pp. 3-7, April 198s.
logic models were not discussed in this paper. Ainong [5] Bereiiji, H.R., An Architecture for Designing
these are Kosko's Fuzzy Cognitive Maps (FChl)[12]. Fuzzy Controllers using Neural Networks, second
These are a graphical representation of tlie causal re- joint technology workshop on neural networks and
lationships between different factors, where tlie weights
fuzzy logic, NASA Johnson Space Center, Hous-
on the links represent the positive or negative causal
relationships. Also, Kosko's Fuzzy Associative Mcm- ton, Texas, 1990.
ories (FAM) [14], Competitive Adaptive Bidircctional [GI Berenji, H.R., Machine Learning in Fuzzy Con-
Memories (CABAM) [13] are very interesting. trol, proceedings of Int. Conf. on Fuzzy Logic
Fuzzy logic controllers and neural network con- & Neural Networks, Iizuka, Fukuoka, Japan, pp.
trollers are complementary, and it is expected that the 231-234, 1990.
aniount of research iiito hybrid approaches will glow
significantly in the next few years, especially in Japan, [7] Berenji, H.R., An Experimental comparison of
where many applications have already been reported machine learning techniques in fuzzy control, 1990
usiiig a combination of these methods. In the U.S., (in preparation).
NASA has taken an active role in integration of t h e [SI Berenji, H.R., Chen, Y.Y., Lee, C.C., Jang, J.S.,
two powerful tecliniques (e.g., First and Second Inter- Murugesan, S., A Hierarchical Approach to De-
iiational Conferences on Neural Networks and Fuzzy signing Approximate Reasoning-based Controllers
Logic a t Johnson Space Center in 19SS and 1990). for Dynamic Physical Systems, Uncertainty in AI
Conference, Boston, MA, 1990.
Discussion
[9] Bernard, J. A., Use of a Rule-Based System for
In this paper, we briefly reviewed soine alternative Process Control, IEEE Control System Magazine,
techniques to classical and modern control theory.
These techniques go beyond linear models on wliich October 19SS.
most of the current control theory is based. Neural net- [lo] Kasai, Y., Morimoto, Y., Electronically controlled
works, fuzzy controllers and their combined models are continuously variable transnlission, in proc. Int.
capable of modeling nonlinear control systems. This is Congress on Transportation Electronics, Dear-
also especially significant when much of the nonlincar- born, Michigan, 1988.
ities stein from the uncertainties in the controller's en-
vironment. We expect that the techniques described in 111 Keller, J.M., & Hunt, D.J., Incorporating Fuzzy
tliis paper will find applications in tlie design of com- Membership Function into the Perceptron Algo-
plex space control systems such as telerobotic assembly rithm, IEEE Trans. Pattern Anal. 8z Machine In-
tasks and rendezvous and docking operations. telligence, PAMI-7, no.6, pp. 693499, 1985.
121 Kosko, B., Fuzzy Cognitive Maps, Int. J. Man-
Acknowledgements Machine Studies, vo1.24,pp.65-75, 1986.
The author is very grateful to Philip Laird, Mark [13] Kosko, B., Competitive adaptive bidirectional as-
Drummond, and Yung-Yaw Clien for reviewing an car- sociative nicniories, ICNN, V01.2, pp. 754766,
lier draft of tliis paper. 1987.
[14] Kosko, B., Fuzzy Associative Memories, In A.
Kandel (ed.), Fuzzy Expert Systems, Addison-
References Wesley, 19S7.
[l] Anderson, C. Learning to Control an Inverted [15] Lea, R. N., Jani, Y., Berenji, H. R., Fuzzy Logic
Pendulum Using Neural Networks, IEEE Control Controller with Reinforcement Learning for Prox-
Systems Magazine, vol. 9, no. 3, pp. 31-37, 19S9. i n i t y Operations and Docking, Fifth IEEE Int.
[2] Antsaklis, P.J., Neural Networks in Control Sys- Synip. on Intelligent Control, Philadelphia, PA,
t e m Magazine, IEEE Control Systems Magazine, Sept. 1990.
April 1990. [lG] Lee, C.C., Self-Learning rule-based controller
[3] Barto, A.G., Sutton, R. S., and Anderson, C.W., employing approximate-reasoning and neural-net
Neuronlike Adaptive Elements That Can Solve concepts, to appear in Int. Journal of Intelligent
Difficult Learning Control Problems, IEEE trans. Systems, 1990.
on Syst., Man, Cybern., vol. SMC-13, pp. 834846, [17] Lee, C. C., Fuzzy Logic in Control Systems: Fuzzy
1983. Logic Controller I and 11, to appear in: IEEE

919
 Trans. on Systems, Man and Cybcrnctics, vol. 20, [24] Sutton, R.S., Learning to Predict by tlie Methods
no. 2, 1990. of Teinporal Differences, Machine Learning (2), 1,
[IS] Lee, C.C., Berenji, H.R., An Intclligcnt Controller pp. 9-44, 1988.
Bascd On Approximate Reasoiling And Reinforcc- [25] Takagi, H., and Hayashi, Isao, Artificial Neural
nicnt Learning, Proc. of IEEE Int. Syinposiuiii on Network driven Fuzzy Reasoning, Int. J. of Ap-
Intelligent Control, Albany, NY, 19S9. proximate Rcasoning, 1990 (to appear).
[19] Manidani, E. H., and Assilian, S., An cxpcri- 1261 Widrow, B., The Original Adaptive Neural Net
nicnt in linguistic synthesis with a fuzzy logic Broom-Balanccr, Int. Symp. Circuits and Sys-
controller, Intcriiatioiial Journal of Man-Machine tenls, pp. 351-357, May 1957.
Studies, Voluiiie 7, no.1, pp. 1-13, 1975. [27] Yager, R., Implementing Fuzzy Logic Controllers
[20] Michie, D., and Chambers, R., Boxes: An experi- Using a Neural Network Framework, Tech. Report
nient in Adaptive Control, in Machine Iiitelligence no. MII-1005, Iona College, New Rochelle, New
2, E. Dale and D. Michie Eds., Olivcr and Boyd, York.
Edinburgh, 1968. [2S] Yasunobu, S. and Miyamoto, S., Automatic Train
[21] Procyk, T.J., and Manidani, E. H., A Linguistic Operation System by Predictive Fuzzy Control,
Self-Organizing Process Controller, Autoinatica, Industrial Application of Fuzzy Control, pp. 1-18,
vol. 15, no.1, pp. 15-30, 1979. 1955.
[22] Rumelhart, D.E., Hinton, G.E., and William, [29] Zadeh, L.A., The concept of a linguistic variable
R.J., Learning Internal Represcntations by Er- and its application to approximate reasoning, In-
ror Propagation, In: Parallel Distributed Process- form. Sci., Part 1: vol. 8, pp. 199-249; Part 2: vol.
ing: Exploring the Microstructure of Cognition, 8, pp. 301-357; Part 3: vol. 9, pp. 43-80, 1975.
Voulnie 1: Foundations, Rumclhart, D.E., and
McClelland, J.L. (eds.), Cambridge, MA, Brad-
ford Books/ MIT Press, 19%.
[23] Selfridge, 0. G., Sutton R.S., and Barto A. G.,
Training and Tracking in Robotics, in Proc. of
the 9th International Joint Conference 011 Arti-
ficial Intelligence, pp. 670472, 1985.

920