International Journal of Computer Applications (0975 – 8887)

Volume 75– No.7, August 2013

Causal Method and Time Series Forecasting model

based on Artificial Neural Network

Benkachcha.S, Benhra.j, El Hassani.H,

Laboratory LISER, ENSEM, Laboratory LISER, ENSEM, École Hassania des Travaux
Km 7 BP 8118 Route El Jadida Km 7 BP 8118 Route El Jadida Publics Km 7 Route El Jadida
Casablanca, Morocco Casablanca, Morocco BP 8108 Casablanca, Maroc

ABSTRACT 2. LITERATURE REVIEW

This article discusses two methods of dealing with demand Quantitative forecasting models can be grouped into two
variability. First a causal method based on multiple regression categories: the time series models and causal methods.
and artificial neural networks have been used. The ANN is Time series analysis tries to determine a model that explains
trained for different structures and the best is retained. the historical demand data and allows extrapolation into the
Secondly a multilayer perceptron model for time series future to provide a forecast in the belief that the demand data
forecasting is proposed. Several learning rules used to adjust represent experience that is repeated over time.
the ANN weights have been evaluated. The results show that
the performances obtained by the two methods are very This category includes naïve method, moving average, trend
similar. The cost criterion is then used to choose the curve analysis, exponential smoothing, and the autoregressive
appropriate model. integrated moving averages (ARIMA) models. These
techniques are appropriate when we can describe the general
General Terms patterns or tendencies, without regard to the factors affecting
Feedforward Neural Networks; Multilayer Perceptron; the variable to be forecast (Kesten C. Green & J. Scott
Levenberg-Marquardt backpropagation algorithm. Armstrong 2012). One of the reasons for its popularity is
Time series forecasting model and causal method. the lower cost. Easy to develop and implement, times series
models are preferred for they have been used in many
Keywords applications such as: Economic Forecasting, Sales
Demand Forecasting, Supply Chain, Time Series, Causal Forecasting, Budgetary Analysis, Stock Market Analysis,
Method, Multiple Regression, Artificial Neural Networks Process and Quality Control and Inventory Studies
(ANN). (GOSASANG V. & all, 2011). On the other hand, the causal
methods search for reasons for demand, and are preferred
1. INTRODUCTION when a set of variables affecting the situation are available
In any production environment, demand forecasting plays an (Armstrong, J. S. 2012). Among the models of this category,
important role for managing integrated logistics system. It multiple linear regression uses a predictive causal model that
provides valuable information for several logistics activities identifies the causal relationships between the predictive
including purchasing, inventory management, and (forecast) variable and a set of predictors (causal factors). For
transportation. In practice there are extensive forecasting example, the customer demand can be predicted through a set
techniques available for anticipating the future. This article of causal factors such as predictors product price, advertising
presents a contribution to improve the quality of forecasts by costs, sales promotion, seasonality, etc. (Chase, 1997).
using Artificial Neural Networks (ANN). It discusses two
methods of dealing with demand variability. First a causal Both kinds of models (time series models and the linear causal
method based on multiple regression and artificial neural methods) are easy to develop and implement. However they
networks have been used. The ANN is trained for different may not be able to capture the nonlinear pattern in data.
structures and the best is retained. Secondly a multilayer Neural Network modelling is a promising alternative to
perceptron model for univariate time series forecasting is overcome these limitations (Chen, K.Y., 2011). Mitrea, C. A.,
proposed. Several learning rules used to adjust the ANN C. K. Lee, M. and Wu, Z. (2009) compared different
weights have been evaluated. The results show that the forecasting methods like Moving Average (MA) and
performances obtained by the two methods are very similar. Autoregressive Integrated Moving Average (ARIMA) with
The cost criterion is then used to choose the appropriate Neural Networks (NN) models as Feed-forward NN and
model. Nonlinear Autoregressive network with eXogenous inputs
(NARX). The results have shown that forecasting with NN
The rest of the paper is organized as follows. Section 2 offers better predictive performances.
reviews the literature in forecasting and the use of Artificial
Neural network in this area. Section 3 presents two prediction After being properly configured and trained by historical
models: causal method and time series model based on neural data, artificial neural networks (ANNs) can be used to
networks. Section 4 discusses the results obtained using this approximate accurately any measurable function. Because
methodology in a case study. Section 5 gives the conclusion they are fast and accurate, several researchers use ANNs to
of the paper. solve problems related to demand forecasting. The essence of
the approach is to learn the patterns hidden in data and make

37
 International Journal of Computer Applications (0975 – 8887)
Volume 75– No.7, August 2013

generalizations of these patterns. ANN provides outstanding The general process responsible for training the network is
results even in the presence of noise or missing information mainly composed of three steps: feed forward the input signal,
(Ortiz-Arroyo D., et all 2005). back propagate the error and adjust the weights.
The back-propagation algorithm try to improve the
3. METHODOLOGY performance of the neural network by reducing the total error
3.1 Feedforward Neural Networks which can be calculated by:
Feedforward neural networks allow only unidirectional signal
1
flow. Furthermore, most feedforward neural networks are
organized in layers and this architecture is often known as
E  o jp  d jp 
2 p j 
MLP (multilayer perceptron) (Wilamowski B. M., 2011). An
example of feedforward neural network is shown in figure 1.
It consists of an input layer, a hidden layer, and an output
Where E is the square error, p the number of applied
layer. patterns, d jp is the desired output for jth neuron when pth

pattern is applied and o jp is the jth neuron output.

Input Hidden Output
Layer Layer Layer
3.2 Regression and multilayer perceptron-
based model
The neural network architecture is composed of input nodes
(corresponding to the independent or predictor variables X1,
X2 and X3), one output node, and an appropriate number of
hidden nodes. The most common approach to select the
Data Input

Output

optimal number of hidden nodes is via experiment or by trial-

Data

and-error (Zhang & all. 1998). We use the same approach to

define the number of neurons in the hidden layer.
One other network design decision includes the selection of
activation functions, the training algorithm, learning rate and
performance measures. For non-linear prediction problem, a
sigmoid function at the hidden layer, and the linear function in
the output layer are generally used. The best training
Fig.1 Basic structure of Multilayer Perceptron algorithm is the Levenberg-Marquardt back-propagation
(Norizan M. & all 2012).

Each node in a neural network is a processing unit which X1

contains a weight and summing function followed by a non-
linearity Fig. 2.

Y
x1 X2
wi1
x2 wi 2
Oi X3
x3 wi 3
….
∑ f
….. Fig.3 Causal method and multi-layer perceptron
wiN
3.3 Time series and multilayer perceptron
xN Fig.2 Single neuron with N inputs. based model
The extrapolative or time series forecasting model is based
only on values of the variable being forecast. Thus for the
The computation related with this neuron is described below: multilayer perceptron used to forecast the time series the
inputs are the past observations of the data series and the
N output is the future value. The MLP performs the following
Oi  f ( wij x j ) function mapping:
j 1
yˆt  f ( yt 1 , yt  2 ,  , yt  n )
Where: Oi is the output of the neuron i , f () the transfer
Where yˆ t is the estimated output, ( yt 1 , yt  2 ,  , yt  n ) is
function, wij is the connection weight between node j and
the training pattern which consist of a fixed number (n) of
node i, and xj the input signal from the node j. lagged observations of the series.

38
 International Journal of Computer Applications (0975 – 8887)
Volume 75– No.7, August 2013

With P observations for the series, we have P-n training Table 2. Data set for causal model based MLP
patterns. The first training pattern is composed of
( y1 , y2 , y3 , y4 , y5 ) as inputs, and ŷ6 as the output N° X1 X2 X3 Y
(Table 2). 1 4949 7409 43500 16793
There is no suggested systematic way to determin the number 2 5369 7903 20209 22455
of the input nodes (n). After several essays this number has 3 6149 9289 47640 25900
been fixed successfully at five nodes for this case study. 4 6655 9914 34563 25591
Table 1. Inputs and estimated output for time series model 5 8114 8193 36581 34396
Assigned data to the input The estimated 6 8447 8711 52206 38676
N° layer output value 7 10472 9453 76788 34608
8 10508 9194 42107 35271
1
9 11534 10149 78902 39132
2 10 12719 10403 73284 46594
3 11 13743 10806 61871 57023
4 12 16168 11557 85265 59720
5 13 16035 11092 28585 62805

6 (Y1 , Y2 , Y3 , Y4 , Y5 ) Y6 14 16112 10979 26921 61905
 15 18861 12117 94457 65963
7 (Y2 , Y3 , Y4 , Y5 , Y6 ) Y7 16 19270 11319 17489 72869

8 (Y3 , Y4 , Y5 , Y6 , Y7 ) Y8 17 21536 12702 99820 71963
18 20783 12419 40152 74915
…. …………………. ……
 19 21917 13265 65490 81030
22 (Y17 , Y18 , Y19 , Y20 , Y21 ) Y22 20 20878 13173 37639 86686
21 21508 13211 19425 97088
The neural network architecture can be shown in figure 4. 22 22261 14070 45300 108739
Other design parameters as training algorithm will be
discussed and selected in the next paragraph.
4.1 Causal model
Neural Network Toolbox provides a complete environment to
Ym design, train, visualize, and simulate neural networks.
For the causal model we use the function newff to create a
Yˆm5
feed forward neural network. The Levenberg-Marquardt back
propagation algorithme is implemented. “trainlm” is a
Ym 1 network training function that updates weights according to
Levenberg-Marquardt optimization (Anandhi V. 2012).
. . . . .

. . . . .

Ym  4

Fig.4 Multilayer perceptron for time series forecasting

model.

4. RESULTS AND DISCUSSIONS

The study is based on a data set used to predict future sales of
a product (Y) based on advertising spending (X1),
promotional expenses (X2) and quarterly sales of its main
competitor (X3).
For this we have 22 observations of three sets of inputs and
output series (Table 2).

Fig.5 Causal Forecast Model Plot Regression

39
 International Journal of Computer Applications (0975 – 8887)
Volume 75– No.7, August 2013

4.2 Time series forecasting model

For the neural network training method, we try three
algorithms to improve network performance:
- The back-propagation (BP) algorithm: It is a gradient
steepest descent method, implemented with traingd function
in MATLAB toolbox.
- Gradient descent with adaptive learning rate back-
propagation (BPA): traingda function in MATLAB toolbox.
- Levenberg-Marquardt back-propagation (LM) : trainlm
function in MATLAB toolbox.

Fig.6 Neural network training for causal forecast

model
The figure below compares the results obtained for different
architectures integrating three, five and seven neurons in the
hidden layer. Fig.8 Plot Regression for time series forecast model
with BP training algorithm

Fig.7 Selection of the number of neurons in the

hidden layer
We can easily conclude that a network with five neurons in Fig.9 Plot Regression for time series forecast model
the hidden layer gives very satisfactory results. with BPA training algorithm

40
 International Journal of Computer Applications (0975 – 8887)
Volume 75– No.7, August 2013

4.3 Comparison between time series

forecasting model and causal model
With the selection of appropriate parameters such as the
number of neurons in each layer and selection of the best
back-propagation algorithm, the two neural models: time
series and causal method, give very similar and satisfactory
results.

Fig.10 Plot Regression for time series forecast model

with LM training algorithm
The plot regression for the three training algorithms in last
figures shown that with the same topology, LM algorithm
gives better solution then those found using BP or BPA
algorithms.
BP algotithm is the most popular method in training neural
networks, however it has a low training efficiency because of
its slow convergence: it has a tendency for oscillation
(Wilamowski B. M. 2011). To overcome this deficiency, the Fig.12 MLP forecasting models : time series model and
BPA algorithm uses dynamic learning rate to increase the causal model
training speed of BP. However even if the BPA algorithm
uses a dynamic learning coefficient, its performance remains However, if we take into consideration the cost of the
less lower than LM, which takes as propagation algorithm a prediction method, the time series model will be chosen since
second order function. In fact the LM algorithm can estimate it takes into consideration only the history of the variable to
the learning rate in each direction of the gradient using predict. In fact, for a causal model, the collection of predictors
Hussian matrix (Wilamowski B. M., Yu H. 2010). is often expensive and can be a source of error in case of
erroneous values or inappropriate extrapolation.

5. CONCLUSION
Demand forecasting plays a crucial role in the supply chain of
today’s company. Among all forecasting methods, neural
networks models are capable of delivering the best results if
they are properly configured. Two approaches based on
multilayer perceptron have been developed to predict demand:
time series model and causal methods. The best training
algorithm is the Levenberg-Marquardt back-propagation
algorithm. The number of hidden layers and the number of
neurons in each layer depends on the chosen method and case
study. With a judicious choice of the architecture and
parameters of the neural network, both approaches have
yielded good results. However, the cost of the prediction
method parameter allows us to prefer the time series model
since we have the same results at a lower cost.

6. REFERENCES
[1] Kesten C. Green, J. Scott Armstrong 2012. Demand
Forecasting: Evidence-based Methods.
https://marketing.wharton.upenn.edu/profile/226/printFri
Fig.11 Demand forecasting with time series model endly.
based on neural network
[2] Gosasang, V., Chan., W. and KIATTISIN, S. 2011. A
Comparison of Traditional and Neural Networks
Forecasting Techniques for Container Throughput at

41
 International Journal of Computer Applications (0975 – 8887)
Volume 75– No.7, August 2013

Bangkok Port. The Asian Journal of Shipping and [8] Wilamowski B. M. 2011. Neural Network Architectures.
Logistics, Vol. 27, N° 3, pp. 463-482. Industrial Electronics Handbook, vol. 5 – Intelligent
Systems, 2nd Edition, chapter 6, pp. 6-1 to 6-17, CRC
[3] Armstrong, J. S. 2012 , Illusions in Regression Analysis, Press.
International Journal of Forecasting, Vol.28, p 689 - 694.
[9] Zhang G., Patuwo, B. E., Hu, M.Y. 1998. Forecasting
[4] Chase, Charles W., Jr., 1997. “Integrating Market with artificial neural networks : The state of the art.
Response Models in Sales Forecasting.” The Journal of International Journal of Forecasting.Vol.14, , p 35–62.
Business Forecasting. Spring: 2, 27.
[10] Norizan M., Maizah H. A., Suhartono, Wan M. A. 2012.
[5] Chen, K.Y. 2011. Combining linear and nonlinear model Forecasting Short Term Load Demand Using Multilayer
in forecasting tourism demand. Expert Systems with Feed-forward (MLFF) Neural Network Model. Applied
Applications, Vol.38, p 10368–10376. Mathematical Sciences, Vol. 6, no. 108, p. 5359 - 5368
[6] Mitrea, C. A., Lee, C. K. M., WuZ. 2009. A Comparison [11] Anandhi V., ManickaChezian R., ParthibanK.T. 2012
between Neural Networks and Traditional Forecasting Forecast of Demand and Supply of Pulpwood using
Methods: A Case Study”. International Journal of Artificial Neural Network. International Journal of
Engineering Business Management, Vol. 1, No. 2, p 19- Computer Science and Telecommunications, Vol.3, Issue
24. 6, June, pp. 35-38
[7] Daniel Ortiz-Arroyo, Morten K. Skov and Quang Huynh, [12] Wilamowski B. M. 2011 Neural Networks Learning.
“Accurate Electricity Load Forecasting With Artificial Industrial Electronics Handbook, vol. 5 – Intelligent
Neural Networks” , Proceedings of the 2005 Systems, 2nd Edition, chapter 11, pp. 11-1 to 11-18, CRC
International Conference on Computational Intelligence Press.
for Modelling, Control and Automation, and
International Conference on Intelligent Agents, Web [13] Wilamowski B. M., Yu H. 2010. Improved Computation
Technologies and Internet Commerce for Levenberg Marquardt Training. IEEE Trans. on
(CIMCAIAWTIC’05) , 2005 Neural Networks, vol. 21, no. 6, pp. 930-937.

42
IJCATM : www.ijcaonline.org