STOCK PRICE PREDICTION USING LSTM

PROJECT REPORT
Submitted by

MANU PRATHAP (113217205033)

SATHISH KUMAR S (113217205047)

SHANMUGA GANESH T (113217205049)

In partial fulfillment of the award of the

degree of

BACHELOR OF TECHNOLOGY
IN
INFORMATION TECHNOLOGY

DEPARTMENT OF INFORMATION TECHNOLOGY

VELAMMAL ENGINEERING COLLEGE

(An Autonomous Institution)

ANNA UNIVERSITY, CHENNAI – 600 025

APRIL, 2021
 VELAMMAL ENGINEERING COLLEGE
DEPARTMENT OF INFORMATION
TECHNOLOGY

BONAFIDE CERTIFICATE

Certified that this project report “STOCK PRICE PREDICTION USING

LSTM” is the bonafide work of MANU PRATHAP (113217205033),

SATHISH KUMAR S (113216205039), SHANMUGA GANESH T

(113216205045) who carried out the project work under my supervision.

Certified further, that to the best of my knowledge the work reported here

does not form part of any other project report or dissertation on the basis of

which a degree or award was conferred on an earlier occasion on this or

any other candidate.

INTERNAL GUIDE HEAD OF DEPARTMENT

Mr. S. RAJ KUMAR Dr. JEEVAA KATIRAVAN

Assistant Professor I Professor & Head
Department of Information Technology, Department of Information Technology,
Velammal Engineering College, Velammal Engineering College,
Chennai. Chennai.

Certified that the candidate was examined in the Viva-Voce Examination held on

__/04/2021.

INTERNAL EXAMINER EXTERNAL EXAMINER

 ACKNOWLEDGEMENT

Behind every achievement lies an unfathomable sea of gratitude to those who

achieved it, without whom it would ever have come into existence. To them we
express our words of gratitude.

We give all the glory and thanks to our almighty GOD for showering upon, the
necessary wisdom and grace for accomplishing this project. We express our
gratitude and thanks to Our Parents first for giving health and sound mind for
completing this project.

First of all we would like to express our deep gratitude to our beloved and
respectable Chairman Thiru M.V. Muthuramalingam and our Chief
Executive Officer Thiru M.V.M. Velmurugan for their kind
encouragement.

We express our deep gratitude to Dr. N. Duraipandian, Principal , Velammal

Engineering College for his helpful attitude towards this project. We wish to
express our sincere thanks and gratitude to Dr. Jeevaa Katiravan, Head of the
Department, Department of Information Technology for motivating and
encouraging in every part of our project.

We express our sincere gratitude to the Project Coordinator Mrs. G.

Prethija, Ms. V. Subbulakshmi, Assistant Professor and the Project
Guide Mr. S. Raj Kumar, Assistant Professor, Department of Information
Technology for their invaluable guidance in shaping this project without
them this project would not have been possible.

Our thanks to all other Faculty and Non-Teaching staff members of our
department for their support and peers for having stood by me and helped me
to complete this project.
 ABSTRACT

In the financial market, there are a large number of indicators used to

describe the change of stock price, which provides a good data basis for our
stock price forecast. Different stocks are affected by different factors due to
their different industry types and regions. Therefore, it is very important to find
a multi factor combination suitable for a particular stock to predict the price of
the stock. This project proposes to use Genetic Algorithm(GA) for feature
selection and develop an optimized Long Short-Term Memory(LSTM) neural
network stock prediction model. Firstly, we use the GA to obtain a factors
importance ranking. Then, the optimal combination of factors is obtained from
this ranking with the method of trial and error. Finally, we use the combination
of optimal factors and the LSTM model for stock prediction. Thorough
empirical studies based upon the Tesla and the S&P 500 stock dataset
demonstrate that the GA-LSTM model can outperform all baseline models for
time series prediction.

Ⅳ
 TABLE OF CONTENTS

CHAPTER TITLE PAGE NO.

NO.
ABSTRACT IV
LIST OF FIGURES VIII
LIST OF ABBREVIATIONS IX
1. INTRODUCTION 1
1.1 OVERVIEW 1
1.2 EXISTING SYSTEM 4
1.2.1 LIMITATIONS 5
1.3 MACHINE LEARNING 5
1.4 DEEP LEARNING 7
9
2. LITERATURE SURVEY
2.1 STOCK PRICE PREDICTION USING LSTM,RNN 9
AND CNN-SLIDING WINDOW MODE
2.2 SVM STOCK SELECTION MODEL USING PCA 9

2.3 STOCK CLOSING PRICE PREDICTION USING 10

MACHINE LEARNING TECHNIQUES
2.4 STOCK PRICE FORECASTING ON 11
INDONESIAN STOCKS.
2.5 DEEP ATTENTIVE LEARNING FOR STOCK
MOVEMENT PREDICTION FROM SOCIAL 12
MEDIA TEXT AND COMPANY
CORRELATIONS

2.6 THE ANALYSIS AND FORECASTING OF 12

STOCK
WITH DEEP LEARNING

3. METHODOLOGY 13
3.1 PROPOSED SYSTEM 12
3.1.1 ADVANTAGES 13
3.2 GENETIC ALGORITHM 15
3.3 DEEP LEARNING 18
3.3.1 RNN 18

V
 3.3.2 LSTM 19
3.3.3 GA-LSTM TWO-STAGE STOCK PRICE 21
PREDICTION MODEL
3.4 MODULES 24
3.4.1 IMPORTING LIBRARIES 24
3.4.2 LOADING DATA 25
3.4.3 READING DATA 25
3.4.4 FEATURE EXTRACTION 26
3.4.5 MODEL BUILDING 26
3.4.6 RESULTS VISUALIZATION 28
3.5 EXPERIMENTS 28
3.5.1 DATASET DESCRIPTION 28
3.5.2 EVALUATION PARAMETERS 29
4. RESULTS AND DISCUSSION 30
4.1 ACCURACY 30
4.2 ACCURACY WITH EACH EPOCHS 31
4.3 ANALYSIS OF TRAINING OF MODEL 32
4.4 FINAL RESULTS 41
5. CONCLUSION AND FUTURE WORK 43
5.1 CONCLUSION 43
5.2 FUTURE WORK 43
6. REFERENCES 44
7. APPENDIX 46

Ⅵ
 LIST OF FIGURES

FIGURE NO. NAME OF THE FIGURE PAGE NO

1 PROPOSED SYSTEM 14
WORKFLOW

2 GENETIC ALGORITHM 16
LIFECYCLE

3 LSTM MODEL DIAGRAM 20

4 FACTOR RANKING ARCH. 23

5 IMPORT LIBRARY CELL 25

6 LOADING DATA CELL 25

7 READING DATA CELL 25
8 MINMAX CELL 26
9 TRAIN TEST SPLIT BLOCK 26
10 MODEL - LSTM BLOCK 27

11 TRAINING OUR MODEL 27

BLOCK

12 VISUALIZATION BLOCK 28

13 DATASET 28

14 CONFUSION MATRIX 29

15 ACCURACY RESULT 30

16 ITERATIONS OF EPOCH 31

17 HISTORICAL VIEW OF 32
CLOSING PRICE

18 TOTAL VOLUME OF STOCK 33

TRADE / DAY

19 ADDITIONAL MOVING 34
AVERAGE OF STOCK

VII
20 GRAPH PLOT BETWEEN 35
MOVING AVERAGE STOCK IN
10, 20, 50 DAYS

21 DAILY RETURN PERCENTAGE 35

22 HISTOGRAM FOR DAILY 36

RETURN USING SEABORN

23 COMPARING GOOGLE WITH 37

ITSELF FOR LINEAR
RELATION

24 DAILY RETURN JOINTLY 37

BETWEEN GOOG & MSFT

25 LINEAR REGRESSION WITH 38

EACH OTHER

26 HISTOGRAM OF PAIRPLOT OF 39
DAILY RETURN

27 CORRELATION WITH EACH 40

OTHER USING SEABORN FOR
DAILY RETURN

28 CORRELATION WITH EACH 40

OTHER USING SEABORN FOR
CLOSING RATE

29 TRAINING PHASE 41

30 OUTPUT (CLOSE VS 41
PREDICTED)

31 PREDICTED WITH CLOSING 42

PRICE

VIII
 LIST OF ABBREVIATIONS

S.NO ABBREVIATION EXPANSION

1. GA Genetic Algorithm

2. LSTM Long Short-Term

Memory
3. RNN Recurrent Neural
Network
4. CNN Convolution Neural
Network
5. SVM Support Vector
Machine
6. ANN Artificial Neural
Network

IX
 CHAPTER 1
INTRODUCTION

1.1 OVERVIEW
With the rapid development of the social economy, the number of
listed companies is increasing, so the stock has become one of the hot
topics in the financial field. The changing trend of stock often affects the
direction of many economic behaviors to a certain extent , so the prediction
of stock price has been paid more and more attention by scholars. The
stock market data has the characteristics of non-linear, high noise,
complexity and timing, etc., so scholars have done a lot of research on the
stock prediction method . The traditional stock prediction method is to
build a linear prediction model based on the historical stock data, Bowden
et al. proposed to use the ARIMA method to build an autoregressive
model to predict stock prices.

Although this method has some advantages in computational efficiency, the

assumption of statistical distribution and stability of the research data
limits their ability to model the nonlinear and non-stationary financial time
series, and the outliers in the research data also have a great impact on the
prediction results. There are many factors affecting stock prices. With the
increasing maturity of statistical techniques in the financial field, financial
scholars have mined a large number of stock market impact factors and
quantified them into specific data for the study of stock change trends.
With the support of massive financial data, it provides the possibility for
the implementation of machine learning algorithms. More and more
researchers begin to use the non-linear prediction model of machine
learning to predict stock prices. Nair et al. proposed a decision tree system

1
based on rough sets. This method combines the advantages of rough sets
and decision trees, but this method is prone to overfitting when dealing
with data sets with a large amount of noise, which will affect the trend of
stock prediction.

In theory, artificial neural networks can learn any nonlinear relationship

and are less disturbed by noise data, so it has been widely used in the field
of time series prediction. Penman , Nottola et al. have respectively carried
out a series of prediction work by using neural networks, and achieved
better results in stock prediction accuracy than decision trees. However,
neural networks are prone to local optimal problems in practical
applications, and support vector machines (SVM) based on structural risk
minimization greatly reduce the possibility of the model falling into local
optimal problems. Cao et al. established the SVM stock prediction model,
which effectively improved the generalization ability of the model.
Ensemble learning is characterized by fast operation rate, strong
anti-interference ability and high accuracy rate. Deng et al. proved in the
experiment that, after parameter optimization, the random forests stock
prediction model has a higher prediction accuracy than SVM.

With the development of artificial intelligence technology, deep learning

has attracted extensive attention due to its excellent performance in
machine translation, voice emotion recognition, image recognition and
other aspects. Compared with the traditional statistical model, the deep
neural network (DNN) can analyze the deep and complex nonlinear
relationship through the layered feature representation, which is suitable
for the multi-factor influence, unstable and complex nonlinear problem of
stock data analysis. Tsantekidis et al. proposed a stock prediction model
based on convolutional neural networks (CNN) and compared it with other

2
classical models to verify the effectiveness of the convolution model in
stock prediction. However, due to the timing of stock data, the
convolutional neural network is not the most suitable neural network
model for stock prediction. Selvin et al. proposed three stock prediction
models based on CNN, recurrent neural network (RNN) and LSTM deep
learning networks respectively, and compared the performance of the three
models by predicting the stock prices of listed companies. Finally, it was
concluded that the LSTM neural network is most suitable for forecasting
the stock market with time series due to its long-term memorability.

For multivariable financial timing prediction, effective feature selection is

very important. Features selection has many benefits, such as:
(i) it reduces the training time of the model;

(ii) it helps in simplifying the complexity of forecasters;

(iii) it improves the accuracy of the model;

(iv) it also avoids over-fitting by eliminating unnecessary variables from

the feature set.

The traditional feature selection methods mainly include filter methods,

embedded methods and wrapper methods. Yu et al. successfully improved
the model prediction accuracy by using PCA for dimension reduction
extraction of feature data combined with SVM model. Qin et al.proposed a
dual-stage attention-based recurrent neural network (DA-RNN) for feature
extraction and sequential prediction. In the first stage, they introduced an
input attention mechanism to adaptively extract relevant driving series at
each time step by referring to the previous encoder hidden state. In the
second stage, they used a temporal attention mechanism to select relevant
encoder hidden states across all time steps. With this dual-stage attention
scheme, their model makes predictions effectively. According to the

3
changes of influence information in different time stages, Zheng et al.
designed a specific attention network and successfully learned the dynamic
influence of the changes of multiple non-predictive time series on the
target series over time.

Although these methods can effectively capture temporal features, they

cannot determine effective multi-factors combination. When the number of
factors increases, the factors tend to have collinearity and interfere with
each other. GA has a good effect in the application of feature selection
problems. The application of population based GA can effectively solve
the problems of noise and collinearity. Therefore, this project proposes the
use of GA for feature selection of multiple factors, and applies it to the
LSTM neural network stock prediction model. Through experimental
comparison, this method achieved remarkable results in improving the
accuracy of stock prediction.

1.2 EXISTING SYSTEM FOR STOCK PREDICTION

In the existing system, we tend to propose that a company’s

performance, in terms of its stock worth movement, is foreseen by internal
communication patterns. to get early warning signals, we tend to believe
that it's vital for patterns in company communication networks to be
detected earlier for the prediction of serious stock worth movement to
avoid attainable adversities that an organization could face within the
securities market in order that stakeholders’ interests are protected the
maximum amount as attainable. Despite the potential importance of such
data regarding corporate communication, very little work has been tired
this vital direction. We attempt to bridge these research gaps by employing

4
a data-mining method to examine the linkage between a firm’s
communication data and its share price. As Enron Corporation’s e-mail
messages constitute the only corpus available to the public, we make use of
Enron’s email corpus as the training and testing data for our proposed
algorithm.

1.2.1 LIMITATIONS

● However accuracy would decrease when setting more levels of stock

market movement.
● The average of prediction accuracies using Decision Tree as the
classifier are 43.44%, 31.92%, and 12.06% for “two levels,” “three
levels,” and “five levels,” respectively.
● These results indicate that the stock price is unpredictable when
traditional classifiers are used.

1.3 MACHINE LEARNING

Machine Learning is the field of study that gives computers the

capability to learn without being explicitly programmed. ML is one of the
most exciting technologies that one would have ever come across. As it is
evident from the name, it gives the computer that makes it more similar to
humans: The ability to learn. Machine learning is actively being used
today, perhaps in many more places than one would expect. Machine
learning implementations are classified into three major categories,
Supervised learning is an algorithm that learns from example data, and
associated target responses. Unsupervised learning is an algorithm learns
from plain examples without any associated response, leaving to the

5
algorithm to determine the data patterns on its own. Reinforcement
learning is when you present the algorithm with examples that lack labels,
as in unsupervised learning. However, you can accompany an example
with positive or negative feedback. Machine learning is used in internet
search engines, email filters to sort out spam, websites to make
personalised recommendations, banking software to detect unusual
transactions, and lots of apps on our phones such as voice
recognition.Machine learning involves computers discovering how they
can perform tasks without being explicitly programmed to do so. It
involves computers learning from data provided so that they carry out
certain tasks. For simple tasks assigned to computers, it is possible to
program algorithms telling the machine how to execute all steps required
to solve the problem at hand on the computer's part, no learning is needed.
For more advanced tasks, it can be challenging for a human to manually
create the needed algorithms. In practice, it can turn out to be more
effective to help the machine develop its own algorithm, rather than having
human programmers specify every needed step. The discipline of machine
learning employs various approaches to teach computers to accomplish
tasks where no fully satisfactory algorithm is available.

In cases where vast numbers of potential answers exist, one approach is to

label some of the correct answers as valid. This can then be used as
training data for the computer to improve the algorithm(s) it uses to
determine correct answers. For example, to train a system for the task of
digital character recognition, theMNISTdataset of handwritten digits has
often been used.Machine learning approaches are traditionally divided into
three broad categories, depending on the nature of the "signal" or
"feedback" available to the learning system:

● Supervised Learning

6
● Unsupervised Learning
● Reinforcement Learning

Other approaches have been developed which don't fit neatly into this
three-fold categorisation, and sometimes more than one is used by the
same machine learning system. For example topic
modelling,dimensionality reduction or meta learning.

1.4 DEEP LEARNING

Deep learning is an artificial intelligence (AI) function that imitates

the workings of the human brain in processing data and creating patterns
for use in decision making. Deep learning is a subset of machine learning
in artificial intelligence that has networks capable of learning unsupervised
from data that is unstructured or unlabeled. Also known as deep neural
learning or deep neural network.

● Deep learning is an AI function that mimics the workings of the

human brain in processing data for use in detecting objects, recognizing
speech, translating languages, and making decisions.
● Deep learning AI is able to learn without human supervision,
drawing from data that is both unstructured and unlabeled.
● Deep learning, a form of machine learning, can be used to help
detect fraud or money laundering, among other functions

7
1.4.1 Deep Learning vs. Machine Learning

One of the most common AI techniques used for processing big data
is machine learning, a self-adaptive algorithm that gets increasingly better
analysis and patterns with experience or with newly added data.

If a digital payments company wanted to detect the occurrence or potential

for fraud in its system, it could employ machine learning tools for this
purpose. The computational algorithm built into a computer model will
process all transactions happening on the digital platform, find patterns in
the data set, and point out any anomaly detected by the pattern.

Deep learning, a subset of machine learning, utilizes a hierarchical level of

artificial neural networks to carry out the process of machine learning. The
artificial neural networks are built like the human brain, with neuron nodes
connected together like a web. While traditional programs build analysis
with data in a linear way, the hierarchical function of deep learning
systems enables machines to process data with a nonlinear approach.

8
 CHAPTER 2

LITERATURE SURVEY

2.1 STOCK PRICE PREDICTION USING LSTM,RNN AND

CNN-SLIDING WINDOW MODE

The proposed method is a model independent approach. Here we are

not fitting the data to a specific model, rather we are identifying the latent
dynamics existing in the data using deep learning architectures. In this
work we use three different deep learning architectures for the price
prediction of NSE listed companies and compare their performance. We
are applying a sliding window approach for predicting future values on a
short term basis.The performance of the models were quantified using
percentage error.

2.2 A SVM STOCK SELECTION MODEL WITHIN PCA

This methodology introduces a machine learning method of support

vector machine to construct a stock selection model, which can do the
nonlinear classification of stocks. However, the accuracy of SVM
classification is very sensitive to the quality of the training set. To avoid
the direct use of complicated and highly dimensional financial ratios, we
bring the principal component analysis (PCA) into SVM model to extract
the low-dimensional and efficient feature information, which improves the
training accuracy and efficiency as well as preserve the features of initial
data. As empirical results show, based on a support vector machine, within
PCA after norm-standardization, the stock selection model achieves the

9
entire accuracy of 75.4464% in the training set and of 61.7925% in the test
set. Further, the PCA-SVM stock selection model contributes the annual
earnings of stock portfolio to outperforming those of A-share index of
Shanghai Stock Exchange, significantly.

2.3 STOCK CLOSING PRICE PREDICTION USING

MACHINE LEARNING TECHNIQUES

With the introduction of artificial intelligence and increased

computational capabilities, programmed methods of prediction have
proved to be more efficient in predicting stock prices. In this work,
Artificial Neural Network and Random Forest techniques have been
utilized for predicting the next day closing price for five companies
belonging to different sectors of operation. The financial data: Open, High,
Low and Close prices of stock are used for creating new variables which
are used as inputs to the model. The models are evaluated using standard
strategic indicators: RMSE and MAPE. The low values of these two
indicators show that the models are efficient in predicting stock closing
price.

10
2.4 STOCK PRICE FORECASTING ON INDONESIAN STOCKS

Forecasting is done by using Artificial Neural Network (ANN) and

for the training, Backpropagation algorithm is used. In this study, stock
price prediction using a neural network with backpropagation algorithm.
ANN is used due to having the ability to perform activities based on past
data, where the data of the past will be studied so as to have the ability to
give a decision on the data that has never been studied. By using
Backpropagation algorithms, network architectures are trained to get the
best architecture. After the training, the best architecture that is obtained is
8: 9: 1. Then, the test was carried out on test data using the best network
architecture and found that the mean squared error (MSE) is equal to
0.1830.

2.5 DEEP ATTENTIVE LEARNING FOR STOCK

MOVEMENT PREDICTION FROM SOCIAL MEDIA TEXT
AND COMPANY CORRELATIONS

We introduce an architecture that achieves a potent blend of chaotic

temporal signals from financial data, social media, and inter-stock
relationships via a graph neural network in a hierarchical temporal fashion.
Through experiments on real-world S&P 500 index data and English
tweets, we show the practical applicability of our model as a tool for
investment decision making and trading.

11
2.6 THE ANALYSIS AND FORECASTING OF STOCK WITH
DEEP LEARNING

This systematic review focuses on Deep Learning Models

implemented for stock market forecasting using technical analysis.
Discussions were made based on four main points of view: predictor
techniques, trading strategies, profitability metrics, and risk
management.This study showed that the LSTM technique is widely
applied in this scenario (73.5%). This work's significant contribution is to
highlight some limitations found in the literature, such as only 35.3% of
the studies analysed profitability, and only two articles implemented risk
management. Therefore, despite the widely explored theme, there are still
interesting open areas for research and development.

12
 CHAPTER 3

METHODOLOGY

3.1 PROPOSED SYSTEM

Accuracy plays an important role in stock market prediction.

Although many algorithms are available for this purpose, selecting the
most accurate one continues to be the fundamental task in getting the best
results. In order to achieve this, in this paper we have compared and
analysed the performance of various available algorithms such as Logistic
regression, SVM, Random Forest, etc. This involves training the
algorithms, executing them, getting the results, comparing various
performance parameters of these algorithms and finally obtaining the most
accurate one.

This project proposes to use Genetic Algorithm(GA) for feature selection

and develop an optimized Long Short-Term Memory(LSTM) neural
network stock prediction model. Firstly, we use the GA to obtain a factors
importance ranking. Then, the optimal combination of factors is obtained
from this ranking with the method of trial and error. Finally, we use the

combination of optimal factors and the LSTM model for stock prediction.

13
 FIG.1 PROPOSED SYSTEM- WORKFLOW

3.1.1 ADVANTAGES

● The successful prediction will maximize the benefit of the

customer.

● In this project, we used stock data of five companies from the

Huge Stock market dataset consisting of data ranging from 2011
to 2021 to train different machine learning algorithms.

● Hence we compared the accuracy of different machine learning

algorithms and came up with the most accurate one.

14
3.2 GENETIC ALGORITHM

GA is an adaptive heuristic search algorithm based on the ideas of

natural selection and genetic evolution, which is widely used to find the
approximate optimal solution of optimization problems with large search
space, and can be effectively used in the selection of optimization features.
GA encodes a potential solution of a problem into an individual, and each
individual is actually an entity with characteristics of chromosomes. The
algorithm calls such a number of individuals together as a population, and
the optimization process of GA is carried out on the population. As the
main carrier of genetic material, chromosomes are a collection of multiple
genes. Its internal expression is a combination of certain genes, which
determines the external expression of individual shape. For example, the
characteristics of black hair are determined by a combination of certain
genes in the chromosome that control this characteristic. Therefore, the
mapping from phenotype to genotype needs to be implemented at the
beginning, i.e., encoding work. Because of the complexity of copying
genetic code, we tend to simplify it, usually in the form of binary strings.
Chromosomes that are closer to the optimal solution will have a better
chance of reproducing. After the initial generation of the population,
according to the principle of survival of the fittest and survival of the
fittest, each generation of evolution produced better and better approximate
solutions. In each generation, individuals are selected according to the
fitness of individuals in the problem domain, and the population
representing the new solution set is generated by combining crossover and
variation with the help of genetic operators. This process will result in the
population evolution, which is like the natural evolution of the population.
Then the population would be more suitable to the environment than the
previous generation. After decoding, the optimal individual in the last

15
generation of the population can be used as the approximate optimal
solution to the problem.

FIG.2 Genetic algorithm lifecycle

GA processing can be divided into seven stages: solu-tion encoding,

initialization, fitness evaluation, termination condition checking, selection,
crossover and mutation .Fig.1 shows the whole process of GA.

16
{α1,α2,...,αn} represents the original feature set. First, it designs a binary
encoding for each chromosome β that represents a potential solution to the
problem, i.e., the binary encoding of each chromosome represents each
feature combination. In the ini-tialization phase, the population size is set
for the population and a random original population {β1,β2,...,βn} is
gen-erated. Then the fitness of each chromosome is calculated according to
the preset fitness function. The fitness function is an evaluation index used
to evaluate the chromosome per-formance. In GA, the definition of fitness
function is a key factor affecting performance. The process of calculat-ing
the fitness function will be used to retain the excellent solution for further
reproduction. High-performing chromo-somes are more likely to be
selected multiple times, while low-performing ones are more likely to be
eliminated. After Several rounds of selection, crossover and mutation
operation,we obtain the optimal chromosomeˆβ. In this paper, we adopt r2
determination coefficient as the fitness function of GA.The determination
coefficient reflects how much percentage of the fluctuation of Y can be
described by the fluctuation of X, i.e., the interpretation degree of the
characteristic variable X to the target value of Y. The determination
coefficient can be defined as follows:

where, the determination coefficient is represented by r2,y is the label

value,ˆy is the predicted value, ̄y is the average value, and the value range
of r2 is [0, 1]. The larger r2 is, the stronger the ability of X to explain Y of
this chromosome is,and the more likely it is to be passed on to the next
generation.The process of chromosome crossing and mutation has great

17
significance to GA. It is beneficial to increase the genetic diversity of the
population to exchange the corresponding part of chromosome chain and
change the gene combination to produce new offspring.

3.3 DEEP LEARNING

Deep learning (also known as deep structured learning or

hierarchical learning) is part of a broader family of Machine learning
methods based on learning data representations, as opposed to task-specific
algorithms. Learning can be supervised, semi-supervised or unsupervised.”

Deep learning is inspired by the human brain and how it perceives

information through interaction of neurons.

It’s a branch of Machine Learning and is implemented through large

Artificial Neural Networks (> 100 layers). Training ANNs for deep
learning requires lots of labeled data as well as huge computing power.

3.3.1 RNN

The idea behind RNNs is to make use of sequential information. In a

traditional neural network we assume that all inputs (and outputs) are
independent of each other. But for many tasks that’s a very bad idea. If you
want to predict the next word in a sentence you better know which words
came before it. RNNs are called recurrent because they perform the same
task for every element of a sequence, with the output being dependent on
the previous computations and you already know that they have a

18
“memory” which captures information about what has been calculated so
far.

3.3.2 LSTM

RNN is a kind of recursive neural network which takes sequence

data as input and performs recursion in sequence evolution direction and
all nodes are connected by chain. Because of its memorability, RNN has
achieved good results in the short sequence model. However, as the length
of input sequence becomes longer, the number of layers in the network will
increase greatly, which will easily cause problems such as gradient
disappearance.

LSTM is a special deep RNN, LSTM greatly enhances the memory

capacity of the model due to its special gate mechanism neural unit
structure, and solves the problem of gradient disappearance caused by
excessively long input sequence in the learning process of traditional cyclic
neural networks. Fig. 2 shows the network structure of LSTM, the LSTM
network saves all the information before each time step in the neural unit
of the current time step, and each neural unit is controlled by the input
gate, forgetting gate and output gate [28]. The input gate is used to control
the input information of the neural unit at the current moment, the
forgetting gate is used to control the historical information stored in the
neural unit at the previous moment, and the output gate is used to control
the output information of the neural unit at the current moment. The
purpose of this design is to allow the LSTM model to selectively
remember more important historical information.

19
 FIG 3. LSTM MODEL DIAGRAM

it= σ(Wi Xt + Ui ht-1 + bi) - (1)

ft= σ(Wf Xt + Uf ht-1 + bf) - (2)

ot= σ(Wo Xt + Uo ht-1 + bo) - (3)

Ct = tanh(Wc Xt + Ucht-1 + bc ) - (4)

Ct= ft × Ct-1 + it × `Ct - (5)

ht = ot × tanh(Ct ) - (6)

where, σ represents the sigmoid activation function in the LSTM network

which is used to speed up the training process, W f , W i , W c and W o are
weight matrices of

forgetting gate, input gate, update gate and output gate respectively, b f , b i
, bc and b o are respectively the bias of forgetting gate, input gate, update
gate and output gate. Finally, the output at the current moment and the
updated cell state at the current moment are calculated.

20
3.3.3 GA-LSTM TWO-STAGE STOCK PRICE PREDICTION
MODEL

This paper mainly proposes a two-stage stock price prediction model

combined with GA and LSTM deep learning network.The experiment is
divided into the following two stages.The first stage is to use GA to sort
the importance of factors. The specific steps are as follows:(i) Binary
encoding of chromosomes, random initialization of the population. We
denote the population of GA using pop as follows:

In the above matrix, each row represents a chromosome (ora feature

selection set), the chromosome length k represents the total number of
features(40 in this paper), number represents the population size(100 in
this paper), the value of a i,j is 0 or 1, 1 represents selection, 0 represents
non-selection,where are non-zero positive.(ii) Taking roulette method for
selection operation. We cal-culate the fitness of each chromosome in the
population. The Probability of each individual being selected is
proportional to the fitness of chromosomes, and the sum of the probability
of chromosome being selected is 1. During the algorithm,the population is
updated once per cycle according to the probability.(iii) The multi-point
intersection method was used to carry out the crossover operation, and the
chromosomes between two individuals will be exchanged, with the
crossover proba-bility set as 0.8. In the algorithm process, a cross operation
is performed on each chromosome in each cycle. The method is to generate

21
a random probability. If the random probability is less than the crossover
probability, the exchange will be carried out, otherwise, there is no
exchange.(iv) The basic bit mutation method is used to carry out mutation
operation. In contemporary individuals, a gene is altered with a small
probability. The probability of variation is set to 0. 003. The algorithm
produces one probability at a time. If the random probability is less than
the crossover prob-ability, variation will be carried out; otherwise, no
variation will be carried out.Loop steps (ii) through (iv) until the iteration
is 100 times.At the end of the algorithm, we generate an optimal
popu-lation close to the optimal solution. In this paper, the total number of
occurrences of each factor in the population is statistically ranked as the
factor importance ranking. The More times the factor appears, the more
important it is.Table 1 shows the specific model parameters.The second
stage of this study is the stage of feature selec-tion optimization of LSTM
stock prediction model. Based On the factor importance ranking obtained
in the previous stage, the top 40, 30, 20, 10 and 5 factors were taken as
input features of the LSTM model. By comparing the prediction results,
the optimal factor combination is determined, and

Table 1

22
 Table 2

FIG 4. FACTOR RANKING ARCHITECTURE

The optimal model is compared with the baseline models to verify the
superiority of the proposed optimization model in improving the model
accuracy.Table 2 shows the specific model parameters. There are three
network layers in the model, namely, the input layer,the hidden layer and
the output layer. The number of neurons in the hidden layer and the output
layer is 128 and 1 respec-tively, and the dropout parameter is set as 0.2 to
randomly remove a part of neurons to avoid overfitting. The
LSTMnetwork time step is set to 5, i.eThis paper takes the historicaldata of
the first five days as input to predict the stock price of the next day. Model
gradient descent optimizer is Adam, andthe number of model iterations is
100. In this paper, the data are divided into a training set and a test set on a

23
scale of 8:2. The First 80% of the data is used for training, while the
remaining20% of the data is used to evaluate the model.The model adopts
mean square error (MSE) as the model evaluation index, and the formula is
as follows,

where, m is the number of samples, yi is the stock price, and ˆyi is the
model forecast stock price. Fig. 3 shows the experimental process of this
study. We obtain 5 multi-factor combinations t40, t30, t20, t10and t5by
ranking the importance of the original feature set{α1,α2,...,αn}. These
multi-factor combinations are used as the input features of LSTM to
predict stock price ˆyt at time t.

3.4 MODULES

3.4.1 IMPORT LIBRARIES

We are going to use numpy for scientific operations, pandas to

modify our dataset, matplotlib to visualize the results, sklearn to scale our
data, and keras to work as a wrapper on low-level libraries like TensorFlow
or Theano high-level neural networks library.

24
 FIG. 5 IMPORT LIBRARY

3.4.2 LOADING DATA

FIG. 6 LOADING DATA

Let’s upload our dataset(If you’re using Google Colab)

If you are not using Google Colab, you can put the data file in the
same code folder.

3.4.3 READING DATA

After uploading our data, we need to make a data frame:

FIG.7 READING DATA CELL

25
3.4.4 FEATURE EXTRACTION

We will use the MinMaxScaler to scale our data between zero

and one. In simpler words, the scaling is converting the numerical
data represented in a wide range into a smaller one.

FIG.8 MINMAX BLOCK

Now we are going to create our training and testing data by calling
our function for each one:

FIG.9 TRAIN TEST SPLIT BLOCK

3.4.5 MODEL BUILDING

First, we initialized our model as a sequential one with 96 units in the

output’s dimensionality. We used return_sequences=True to make the
LSTM layer with three-dimensional input and input_shape to shape our
dataset. Making the dropout fraction 0.2 drops 20% of the layers. Finally,
we added a dense layer with a value of 1 because we want to output one
value.

26
 FIG.10 MODEL - LSTM BLOCK

After that, we want to reshape our feature for the LSTM layer, because it is
sequential_3 which is expecting 3 dimensions, not 2:

FIG.11 TRAINING OUR MODEL BLOCK

27
 3.4.6 RESULTS VISUALIZATION

The last step is to visualize our data.

FIG.12 VISUALIZATION BLOCK

3.5 EXPERIMENTS

3.5.1 DATASET DESCRIPTION

If you take a look at the dataset, you need to know that the “open”
column represents the opening price for the stock at that “date” column,

FIG.13 DATASET

28
 and the “close” column is the closing price on that day. The “High” column
represents the highest price reached that day, and the “Low” column
represents the lowest price.

3.5.2 EVALUATION PARAMETERS

To evaluate the performance of Model, we have considered the following

● Accuracy
● Classification Report
● Confusion Matrix

FIG.14-CONFUSION MATRIX

29
 CHAPTER 4

RESULTS AND DISCUSSION

Classification results demonstrate that the capability of automatic

feature extraction with deep learning models plays an essential role in the
accurate prediction of a company’s stock. Our proposed model produced
more accurate results as compared to existing benchmarks with an
accuracy of around 80-90%.

4.1 ACCURACY

Fig.15 ACCURACY RESULT

30
4.2 ACCURACY WITH EACH EPOCHS

FIG.16 ITERATIONS OF EPOCHS

31
44.3 ANALYSIS OF TRAINING OF MODEL

FIG.17 HISTORICAL VIEW OF CLOSING PRICE

32
FIG.18 TOTAL VOLUME OF STOCK TRADE PER DAY

33
FIG.19 ADDITIONAL MOVING AVERAGE OF STOCK

34
FIG.20 GRAPH PLOT BETWEEN MOVING AVERAGE STOCK IN 10, 20, 50
DAYS

FIG.21 DAILY RETURN PERCENTAGE

35
FIG.22 HISTOGRAM FOR DAILY RETURN USING SEABORN

36
 FIG.23 COMPARING GOOGLE WITH ITSELF FOR LINEAR RELATION

FIG.24 DAILY RETURN JOINTLY BETWEEN GOOG & MSFT

37
FIG.25 LINEAR REGRESSION WITH EACH OTHER

38
FIG.26 HISTOGRAM OF PAIRPLOT OF DAILY RETURN

39
FIG.27 CORRELATION WITH EACH OTHER USING SEABORN FOR DAILY
RETURN

FIG.28 CORRELATION WITH EACH OTHER USING SEABORN FOR CLOSING

RATE

40
FIG.29 TRAINING PHASE

4.4 FINAL RESULTS

FIG.30 OUTPUT (CLOSE VS PREDICTED)

41
FIG.31 PREDICTED WITH CLOSING PRICE

42
 CHAPTER 5

CONCLUSION AND FUTURE WORK

5.1 CONCLUSION
With the help of Genetic Algorithm and LSTM, we were able to
predict the stock price of a company with an accuracy over 90 %. Thus
making it one of the best methods to predict the stock price of any
company’s stocks.

5.2 FUTURE WORK

● We can further increase model size, tweak the hyperparameters, use

Bidirectional LSTM and test if the model performs better.
● We can further optimize it by upgrading simple LSTM to LSTM with
peephole architecture.
● We can combine other machine learning algorithms such as Random
Forest, SVM along with LSTM to improve the efficiency of the model.

43
 CHAPTER 6
REFERENCES

[1] A. Tsantekidis, N. Passalis, A. Tefas, J. Kanniainen, M. Gabbouj, and

A. Iosifidis, ‘‘Forecasting stock prices from the limit order book using
convolutional neural networks,’’ in Proc. IEEE 19th Conf. Bus. Informat.
(CBI), Beijing, China, Jul. 2017, pp. 10–15.

[2] S. Selvin, R. Vinayakumar, E. A. Gopalakrishnan, V. K. Menon, and K.

P. Soman, ‘‘Stock price prediction using LSTM, RNN and CNN-sliding
window model,’’ in Proc. Int. Conf. Adv. Comput., Commun. Informat.
(ICACCI), Beijing, China, Sep. 2017, pp. 234–239.

[3] Y. Kim, J.-H. Roh, and H. Kim, ‘‘Early forecasting of rice blast disease
using long short-term memory recurrent neural networks,’’ Sustainability,
vol. 10, no. 2, p. 34, Dec. 2017.

[4] D. Wang and J. Fang, ‘‘Research on optimization of big data

construction engineering quality management based on
RNN-LSTM,’’ Complexity, vol. 15, no. 2, pp. 15–20, Jun. 2018.

[5] J. Deng and L. Li, ‘‘Application of parameter optimization stochastic

forest in stock prediction,’’ Software, vol. 41, no. 1, pp. 178–182, Jan.
2020.

[6] D. Lv, D. Wang, M. Li, and Y. Xiang, ‘‘DNN models based on

dimensionality reduction for stock trading,’’ Intell. Data Anal., vol. 24, no.
1, pp. 19–45, Feb. 2020.

44
[7] J. Hu and W. Zheng, ‘‘Multistage attention network for multivariate
time series prediction,’’ Neurocomputing, vol. 18, no. 383, pp. 122–137,
Mar. 2020.
[8] U. F. Siddiqi, S. M. Sait, and O. Kaynak, ‘‘Genetic algorithm for the
mutual information-based feature selection in univariate time series data,’’
IEEE Access, vol. 8, pp. 9597–9609, 2020.

45
 CHAPTER 7
APPENDIX

import pandas as pd
import numpy as np

import matplotlib.pyplot as plt

import seaborn as sns
sns.set_style('whitegrid')
plt.style.use("fivethirtyeight")
# %matplotlib inline

# For reading stock data from yahoo

from pandas_datareader.data import DataReader

# For time stamps

from datetime import datetime

# The tech stocks we'll use for this analysis

tech_list = ['AAPL', 'GOOG', 'MSFT', 'AMZN']

# Set up End and Start times for data grab

end = datetime.now()
start = datetime(end.year - 1, end.month, end.day)

46
#For loop for grabbing yahoo finance data and setting
as a dataframe
for stock in tech_list:
# Set DataFrame as the Stock Ticker
globals()[stock] = DataReader(stock, 'yahoo', start,
end)

# for company, company_name in zip(company_list,

tech_list):
# company["company_name"] = company_name

company_list = [AAPL, GOOG, MSFT, AMZN]

company_name = ["APPLE", "GOOGLE", "MICROSOFT",
"AMAZON"]

for company, com_name in zip(company_list,

company_name):
company["company_name"] = com_name

df = pd.concat(company_list, axis=0)
df

# Summary Stats
AAPL.describe()

47
# General info
AAPL.info()

# Let's see a historical view of the closing price

plt.figure(figsize=(12, 8))
plt.subplots_adjust(top=1.25, bottom=1.2)

for i, company in enumerate(company_list, 1):

plt.subplot(2, 2, i)
company['Adj Close'].plot()
plt.ylabel('Adj Close')
plt.xlabel(None)
plt.title(f"{tech_list[i - 1]}")

# Now let's plot the total volume of stock being traded

each day
plt.figure(figsize=(12, 8))
plt.subplots_adjust(top=1.25, bottom=1.2)

for i, company in enumerate(company_list, 1):

plt.subplot(2, 2, i)
company['Volume'].plot()
plt.ylabel('Volume')
plt.xlabel(None)
plt.title(f"{tech_list[i - 1]}")

48
ma_day = [10, 20, 50]

for ma in ma_day:
for company in company_list:
column_name = f"MA for {ma} days"
company[column_name] = company['Adj
Close'].rolling(ma).mean()

print(GOOG.columns)

df.groupby("company_name").hist(figsize=(12, 12));

fig, axes = plt.subplots(nrows=2, ncols=2)

fig.set_figheight(8)
fig.set_figwidth(15)

AAPL[['Adj Close', 'MA for 10 days', 'MA for 20 days',

'MA for 50 days']].plot(ax=axes[0,0])
axes[0,0].set_title('APPLE')

GOOG[['Adj Close', 'MA for 10 days', 'MA for 20 days',

'MA for 50 days']].plot(ax=axes[0,1])

49
axes[0,1].set_title('GOOGLE')

MSFT[['Adj Close', 'MA for 10 days', 'MA for 20 days',

'MA for 50 days']].plot(ax=axes[1,0])
axes[1,0].set_title('MICROSOFT')

AMZN[['Adj Close', 'MA for 10 days', 'MA for 20 days',

'MA for 50 days']].plot(ax=axes[1,1])
axes[1,1].set_title('AMAZON')

fig.tight_layout()

# We'll use pct_change to find the percent change for

each day
for company in company_list:
company['Daily Return'] = company['Adj
Close'].pct_change()

# Then we'll plot the daily return percentage

fig, axes = plt.subplots(nrows=2, ncols=2)
fig.set_figheight(8)
fig.set_figwidth(15)

AAPL['Daily Return'].plot(ax=axes[0,0], legend=True,

linestyle='--', marker='o')
axes[0,0].set_title('APPLE')

GOOG['Daily Return'].plot(ax=axes[0,1], legend=True,

linestyle='--', marker='o')

50
axes[0,1].set_title('GOOGLE')

MSFT['Daily Return'].plot(ax=axes[1,0], legend=True,

linestyle='--', marker='o')
axes[1,0].set_title('MICROSOFT')

AMZN['Daily Return'].plot(ax=axes[1,1], legend=True,

linestyle='--', marker='o')
axes[1,1].set_title('AMAZON')

fig.tight_layout()

# Note the use of dropna() here, otherwise the NaN

values can't be read by seaborn
plt.figure(figsize=(12, 12))

for i, company in enumerate(company_list, 1):

plt.subplot(2, 2, i)
sns.distplot(company['Daily Return'].dropna(),
bins=100, color='purple')
plt.ylabel('Daily Return')
plt.title(f'{company_name[i - 1]}')
# Could have also done:
#AAPL['Daily Return'].hist()

# Grab all the closing prices for the tech stock list

51
into one DataFrame
closing_df = DataReader(tech_list, 'yahoo', start,
end)['Adj Close']

# Let's take a quick look

closing_df.head()

# Make a new tech returns DataFrame

tech_rets = closing_df.pct_change()
tech_rets.head()

# Comparing Google to itself should show a perfectly

linear relationship
sns.jointplot('GOOG', 'GOOG', tech_rets,
kind='scatter', color='seagreen')

# We'll use joinplot to compare the daily returns of

Google and Microsoft
sns.jointplot('GOOG', 'MSFT', tech_rets,
kind='scatter')

sns.pairplot(tech_rets, kind='reg')

# Set up our figure by naming it returns_fig, call

PairPLot on the DataFrame

52
return_fig = sns.PairGrid(tech_rets.dropna())

# Using map_upper we can specify what the upper

triangle will look like.
return_fig.map_upper(plt.scatter, color='purple')

# We can also define the lower triangle in the figure,

including the plot type (kde)
# or the color map (BluePurple)
return_fig.map_lower(sns.kdeplot, cmap='cool_d')

# Finally we'll define the diagonal as a series of

histogram plots of the daily return
return_fig.map_diag(plt.hist, bins=30)

# Set up our figure by naming it returns_fig, call

PairPLot on the DataFrame
returns_fig = sns.PairGrid(closing_df)

# Using map_upper we can specify what the upper

triangle will look like.
returns_fig.map_upper(plt.scatter,color='purple')

# We can also define the lower triangle in the figure,

including the plot type (kde) or the color map
(BluePurple)
returns_fig.map_lower(sns.kdeplot,cmap='cool_d')

# Finally we'll define the diagonal as a series of

53
histogram plots of the daily return
returns_fig.map_diag(plt.hist,bins=30)

# Let's go ahead and use sebron for a quick correlation

plot for the daily returns
sns.heatmap(tech_rets.corr(), annot=True,
cmap='summer')

sns.heatmap(closing_df.corr(), annot=True,
cmap='summer')

# Let's start by defining a new DataFrame as a cleaner

version of the original tech_rets DataFrame
rets = tech_rets.dropna()

area = np.pi*20

plt.figure(figsize=(12, 10))
plt.scatter(rets.mean(), rets.std(), s=area)
plt.xlabel('Expected return')
plt.ylabel('Risk')

for label, x, y in zip(rets.columns, rets.mean(),

rets.std()):
plt.annotate(label, xy=(x, y), xytext=(50, 50),
textcoords='offset points', ha='right', va='bottom',
arrowprops=dict(arrowstyle='-',
color='blue', connectionstyle='arc3,rad=-0.3'))

54
# Get the stock quote
df = DataReader('AAPL', data_source='yahoo',
start='2012-01-01', end=datetime.now())
# Show teh data
df

plt.figure(figsize=(16,8))
plt.title('Close Price History')
plt.plot(df['Close'])
plt.xlabel('Date', fontsize=18)
plt.ylabel('Close Price USD ($)', fontsize=18)
plt.show()

# Create a new dataframe with only the 'Close column

data = df.filter(['Close'])
# Convert the dataframe to a numpy array
dataset = data.values
# Get the number of rows to train the model on
training_data_len = int(np.ceil( len(dataset) * .95 ))

training_data_len

# Scale the data

from sklearn.preprocessing import MinMaxScaler

scaler = MinMaxScaler(feature_range=(0,1))
scaled_data = scaler.fit_transform(dataset)

55
scaled_data

# Create the training data set

# Create the scaled training data set
train_data = scaled_data[0:int(training_data_len), :]
# Split the data into x_train and y_train data sets
x_train = []
y_train = []

for i in range(60, len(train_data)):

x_train.append(train_data[i-60:i, 0])
y_train.append(train_data[i, 0])
if i<= 61:
print(x_train)
print(y_train)
print()

# Convert the x_train and y_train to numpy arrays

x_train, y_train = np.array(x_train), np.array(y_train)

# Reshape the data

x_train = np.reshape(x_train, (x_train.shape[0],
x_train.shape[1], 1))
# x_train.shape

from keras.models import Sequential

from keras.layers import Dense, LSTM

56
# Build the LSTM model
model = Sequential()
model.add(LSTM(128, return_sequences=True, input_shape=
(x_train.shape[1], 1)))
model.add(LSTM(64, return_sequences=False))
model.add(Dense(25))
model.add(Dense(1))

# Compile the model

model.compile(optimizer='adam',
loss='mean_squared_error')

# Train the model

model.fit(x_train, y_train, batch_size=1, epochs=1)

# Create the testing data set

# Create a new array containing scaled values from
index 1543 to 2002
test_data = scaled_data[training_data_len - 60: , :]
# Create the data sets x_test and y_test
x_test = []
y_test = dataset[training_data_len:, :]
for i in range(60, len(test_data)):
x_test.append(test_data[i-60:i, 0])

# Convert the data to a numpy array

x_test = np.array(x_test)

57
# Reshape the data
x_test = np.reshape(x_test, (x_test.shape[0],
x_test.shape[1], 1 ))

# Get the models predicted price values

predictions = model.predict(x_test)
predictions = scaler.inverse_transform(predictions)

# Get the root mean squared error (RMSE)

rmse = np.sqrt(np.mean(((predictions - y_test) ** 2)))
rmse

# Plot the data

train = data[:training_data_len]
valid = data[training_data_len:]
valid['Predictions'] = predictions
# Visualize the data
plt.figure(figsize=(16,8))
plt.title('Model')
plt.xlabel('Date', fontsize=18)
plt.ylabel('Close Price USD ($)', fontsize=18)
plt.plot(train['Close'])
plt.plot(valid[['Close', 'Predictions']])
plt.legend(['Train', 'Val', 'Predictions'], loc='lower
right')
plt.show()

# Show the valid and predicted prices

58
valid

59