Data Augmentation On Plant Leaf Disease Image

Dataset Using Image Manipulation and Deep

Learning Techniques
Arun Pandian J G Geetharamani Annette B
Dept. of Computer Science and Dept. of Mathematics, Dept. of Management Studies,
Engineering, University College of Engineering, M.A.M. College of Engg. & Tech.,
M.A.M. College of Engg. & Tech., Anna University - BIT Campus, Tiruchirappalli, India
Tiruchirappalli, India Tiruchirappalli, India [email protected]
[email protected] [email protected]

Abstract—A large volume of data will increase the noise injection techniques. These techniques create
performance of machine learning algorithms and avoid augmented images using any one or combination of
overfitting problems. Collecting a large amount of training multiple image formatting methods [26].
data in the agricultural field for designing plant leaf disease
detection and diagnosis model is a highly challenging task Generative Adversarial Networks (GANs) and Neural
which takes more time and resources. Data augmentation Style Transfer (NST) techniques are advanced image
increases the diversity of training data for machine learning transformation techniques and can create new augmented
algorithms without collecting new data. In this article, images using deep neural networks [18, 20]. Generative
augmented plant leaf disease datasets was developed using Adversarial Networks are the class of generative models,
basic image manipulation and deep learning based image
augmentation techniques such as Image flipping, cropping,
which create new unique data from the existing data in the
rotation, color transformation, PCA color augmentation, dataset. Generator and discriminator are the two major
noise injection, Generative Adversarial Networks (GANs) neural network parts in the GAN [21]. In the training
and Neural Style Transfer (NST) techniques. Performance of process, the Generator produces new data and tries to
the data augmentation techniques was studied using state-of- convince the Discriminator, and the Discriminator tries to
the-art transfer learning techniques, for instance, VGG16, compare the original and generated data. The training
ResNet, and InceptionV3. An extensive simulation shows process will not be complete until the classification
that the augmented dataset using GAN and NST techniques accuracy of the Discriminator gets very less.
achieves better accuracy than the original dataset using a
basic image manipulation based augmented dataset. Deep Convolutional Generative Adversarial Networks
Furthermore, a combination of Deep learning, Color, and (DCGANs) is a class of unsupervised learning algorithms
Position augmentation dataset gives the maximum and uses convolutional neural networks (CNNs) to build
classification performance than all other datasets. the generator and discriminator [4]. DCGAN was used in
Keywords- Data augmentation, Deep Learning, Generative several applications in the computer vision field to create
Adversarial Networks, Image manipulation, Neural Style unique images. The performance of the DCGAN is better
Transfer. than the traditional GAN network.
I. INTRODUCTION Moreover, Wasserstein GANs (WGANs) is another
Identification Plant leaf diseases are most important to well-known GAN based learning approach [5]. The
increase the farmer’s benefits in terms of crop yield and WGAN is a modified version of DCGAN. It uses a linear
growth. Continuous monitoring of diseases in plant leaf is activation function instead of sigmoid in the output layer
more difficult and expensive to implement in real-time [1]. and Wasserstein loss function to calculate the loss values
In recent years, several artificial intelligence methods have of the training process.
proposed different techniques to detect and diagnose plant On the other hand, Neural Style Transfer (NST) is an
leaf diseases using image processing techniques [2]. advanced image transformation technique. It uses deep
Typically, Deep Convolutional Neural Network neural networks to perform the image transformation [6].
(DCNN) plays an essential role in agriculture and medical The NST uses two input images to generate output images,
image classification and detection [3, 9]. The DCNN needs named as a content image and style image. It takes the
a large volume of training data and a substantial input information from the content images and uses the
computational resource. Advances in Graphical Processing design patterns of style images to create an output image.
Units (GPUs) can solve the high-performance computation Mainly, it transforms the style into the content image to
requirement issues. generate a new artistic output image [23, 24]. The NSTs
are mostly used to create artificial paintings and designs.
Data augmentation techniques can increase the
quantity of data in a dataset without collecting new data. The remaining part of this article is organizing as
The data augmentation techniques create new data from follows; Section 2 presents a detailed literature survey of
the original training data using some basic manipulation deep learning based data augmentation techniques. Section
and advanced image transformation techniques [28]. The 3 describes the original plant leaf disease dataset for plant
most common image manipulation techniques are Image leaf disease detection. Section 4 discusses the use of basic
flipping, cropping, rotation, color transformation, and image manipulation techniques to create an augmented

978-1-7281-4392-7/19/$31.00 2019
c IEEE 199
 dataset. Section 5 discusses the implementation of Deep generator and the discriminator to solve mode collapse
learning based data augmentation approaches. Section 6 issues. The authors in [17] review the applications and
provides the performance comparison of the developed limitations of GANs in medical imaging applications.
datasets. As a final point, section 6 presents the conclusion
In [19], the authors describe the challenges and
and future works.
drawbacks of training GANs for the data augmentation
II. LITERATURE SURVEY process. In [6], the authors propose an NST technique to
transfer the image style to the input image using neural
The most crucial objective of this survey is to illustrate
networks.
the various deep learning techniques that are used to
generate augmented images and the study of the The authors in [22] have developed an NST algorithm
significance of the augmentation process. to generate real-world images. They use the instance
normalization function for the training process instead of
In [10], the authors proposed a liver lesion
batch normalization, which achieves better performance
classification using GAN data augmentation. The
than standard NST technique. Authors in [8] developed the
augmented dataset based classification algorithm achieved
fastest NST algorithm to create augmented images. The
the sensitivity and specificity of 78.6% and 88.4%,
drawback of the NST algorithm was limited style transfer.
respectively. The classification algorithm using the GAN
data augmentation attained a sensitivity of 85.7% and a III. ORIGINAL PLANT LEAF DISEASE DATASET
specificity of 92.4%. These results are more excellent than
The original plant leaf disease dataset was download
the original dataset.
from the Mendeley open data repository [34]. The dataset
Similarly, the authors in [27] have proposed a liver contains 38 different healthy and diseased plant leaf
lesion classification model using GAN-based data classes of 54305 color images.
augmentation to achieve improved performance. In [29],
There are 13 different plant leaves used in the dataset.
the author proposed a DCNN model for liver lesion
The number of images in every single class is not in a
classification problem using DGAN based data
uniform range, and it shows that the data set is imbalanced.
augmentation technique. The generated images are
For instance, the orange Huanglongbing class contains
evaluated using a visual Turing test [11]. Likewise, the
5507 images, and the Healthy potato class consists of only
authors in [30] created DCGAN and WGAN based
152 images. The imbalanced dataset may affect the
augmented images for brain tumor MRI datasets. The
performance of the classification model.
authors evaluate the quality of the generated images using
several performance metrics. Figure 1 shows the sample images from the original
dataset.
The authors in [12] developed a new GAN
architecture, which improved the output image quality of
the image-to-image translation process. The architecture
was named CycleGAN, and it uses the Cycle-Consistency
loss function to stabilize the training process of the
networks.
Another architecture proposed by the authors in [7]
named as Progressively Growing GANs, uses progressive
resolution complexity to train the networks. This approach
train the model with progressive resolution. For instance,
the resolution range starts from the input size of 4 × 4 to 8
× 8, and so on until output resolution of 1024 × 1024 is
achieved.
In [16], the authors compared the performance of
several loss functions of generator and discriminator. The
authors conclude that most of the loss functions give a
similar performance with optimized hyperparameters.
In [25], the authors used GANs to create realistic data
from their simulated data. CycleGAN based augmented
dataset was designed to avoid class imbalance issues by
the authors in [14]. It created an augmented facial Fig 1. Sample data from the original dataset
expression database to recognize seven different facial
emotions. Similarly, the performance of the CycleGANs The data augmentation techniques are introduced to
was tested by the authors in [13] for the facial expression handle the imbalanced dataset. The basic image
recognition task. manipulation based augmentation techniques to solve the
data imbalance issues discussed in the next section.
Another GAN architecture proposed by the authors in
[15] is Conditional GANs for image augmentation. The Figure 2 shows the imbalanced structure of the original
Conditional GANs are using conditional vector in the dataset.

200 9th International Conference on Advanced Computing (IACC)

 The size of the augmented dataset was 76000 images.
The augmented dataset was shared in the kaggle open data
repository [32].
V. DATA AUGMENTATION USING DEEP
LEARNING TECHNIQUES
Deep Convolutional Generative Adversarial Networks,
Wasserstein GAN, and Neural Style Transfer are the most
common Deep learning based data augmentation
techniques. The standard DCGAN and WGAN were
trained using NVIDIA DGX-1 V100 deep learning server
with epochs of 1000 and batch size of 64.
The DCGAN and WGAN were generated 32073
augmented images to enhance the size of the original
dataset. It balanced the dataset by the individual class size
of 2000 images. Figure 4 illustrates the DCGAN and
WGAN based sample augmented images. The augmented
dataset was shared in the kaggle open data repository [33].

Fig 2. Structure of the original dataset

IV. DATA AUGMENTATION USING BASIC

MANIPULATION TECHNIQUES
The most common image manipulation techniques are
Image flipping, cropping, rotation, color transformation,
principal component analysis (PCA), and noise injection.
The Basic image manipulation techniques creates 32073
augmented images and it balanced each class size with
2000 images in the plant leaf disease dataset. Figure 3
shows sample augmented images using basic image
manipulation techniques.

Fig 4. Sample augmented data using DCGAN and WGAN

The Neural Style Transfer (NST) augmentation

technique creates 6038 images. The convolutional neural
network was used to design the Neural Style Transfer
model. The model was trained using the training epochs of
3000 and a minibatch size of 64.
The augmentated data equalized the individual class
size as 1000 images. The augmented dataset size was
38,000 images. The augmented dataset was shared in the
kaggle open data repository [35].
Figure 5 shows the random samples from the
augmented dataset using the Neural Style Transfer
technique.

Fig 3. Sample augmented data using basic manipulation

9th International Conference on Advanced Computing (IACC) 201

 Fig 7. Performance of different transfer learning techniques using the
basic image manipulation based augmented dataset

Furthermore, the DCGAN and WGAN augmented

dataset based classification models are trained using the
value of the same hyperparameter. After the training
process, the models are tested using unseen testing images.
The testing results show that the performance of the
classification models using the DCGAN and WGAN
Fig 5. Sample augmented data using NST augmented dataset gives a superior performance than the
previous datasets. Figure 8 shows the performance of the
VI. RESULTS AND DISCUSSIONS DCGAN and WGAN augmented dataset.
At first, the original dataset was used to train the
models for plant leaf disease identification with different
state-of-the-art transfer learning techniques such as
VGG16, ResNet, and InceptionV3.
The dataset divided into training, validation, and
testing set at 90%, 5%, and 5%, respectively. Figure 6
shows the classification accuracy of the models using
original dataset.

Fig 8. Performance of different transfer learning techniques using the

DCGAN and WGAN augmented dataset

Likewise, the pre-trained models are trained using the

NST augmented dataset. The testing result shows that the
NST augmented dataset gives better accuracy than the
basic manipulation dataset and original dataset. However,
the performance is quite less than the DCGAN and
WGAN augmented dataset. Figure 9 shows the testing
Fig 6. Performance of different transfer learning techniques using the results of the dataset.
original dataset

Moreover, the basic image manipulation technique

based dataset was used to train the pre-trained models with
similar hyperparameters value. The testing result shows
that the augmented dataset gives a better classification
performance than the original dataset.
Figure 7 shows the performance of the different
models using the basic image manipulation based
augmented dataset.

Fig 9. Performance of different transfer learning techniques using the

NST augmented dataset

Finally, the combined dataset was created by using all

the augmentation techniques such as basic manipulation,

202 9th International Conference on Advanced Computing (IACC)

DCGAN and WGAN, and NST. The size of the dataset [4] Alec Radford and Luke Metz and SoumithChintala, Unsupervised
Representation Learning with Deep Convolutional Generative
was 228000. The classification accuracy of the pre-trained
Adversarial Networks, arXiv, 2015
models using the dataset was much higher than other
datasets. Figure 10 shows the classification accuracy of the [5] Martin Arjovsky and SoumithChintala and Léon Bottou, Wasserstein
GAN, arXiv, 2017
pre-trained models using the dataset.
[6] Leon A. Gatys and Alexander S. Ecker and Matthias Bethge, A
Neural Algorithm of Artistic Style, arXiv, 2015
[7] Tero K, Timo A, Samuli L, Jaakko L. Progressive growing of GANs
for improved quality, stability, and variation. CoRR,
abs/1710.10196, 2017.
[8] Justin J, Alexandre A, Li FF. Perceptual losses for real-time style
transfer and super-resolution. ECCV. 2016;2016:694–711.
[9] In Y, Paul SB, Ekta W. Generative adversarial network in medical
imaging: a review. arXiv preprint. 2018.
[10] Maayan F-A, Eyal K, Jacob G, Hayit G. GAN-based data
augmentation for improved liver lesion classification. arXiv
preprint. 2018.

Fig 10. Performance of different transfer learning techniques using the [11] Laurens M, Geoffrey H. Visualizing data using t-SNE. J Mach Learn
combined dataset Res. 2008;9:2431–56.
[12] Jun-Yan Z, Taesung P, Phillip I, Alexei AE. Unpaired image-to-
The experimental result illustrates that the combination image translation using cycle-consistent adversarial networks. In:
of different data augmentation techniques based combined International conference on cmoputer vision (ICCV ), 2017
dataset gives superior performance than the original [13] Xinyue Z, Yifan L, Zengchang Q, Jiahong L. Emotion classification
dataset and individual augmentation techniques based with data augmentation using generative adver-sarial networks.
datasets. Also, the performance of deep learning based CoRR, vol. abs/1711.00648. 2017.
dataset is better than the basic manipulation techniques. [14] Goodfellow IJ, Erhan D, Carrier PL, Courville A, Mirza M, Hamner
Finally, the combined dataset was shared in the kaggle B, Cukierski W, Tang Y, Thaler D, Lee DH, et al. Chal-lenges in
open data repository [31]. representation learning: A report on three machine learning contests.
In: NIPS. Berlin: Springer; 2013. p.117–24.
VII. CONCLUSION
[15] Mehdi M, Simon O. Conditional generative adversarial nets. arXiv
In this research, four different image augmented preprint. 2014
datasets are proposed to solve the classification problems [16] Mario L, Karol K, Marcin M, Olivier B, Sylvain G. Are GANs
in plant leaf disease detection. The datasets are Basic created equal? A large-scale study. arXiv preprint. 2018.
image manipulation augmented dataset, DCGAN and [17] Swee KL, Yi L, Ngoc-Trung T, Ngai-Man C, Gemma R, Yuval E.
WGAN augmented dataset, NST augmented dataset, and a DOPING: generative data augmentation for unsuper-vised anomaly
combined augmented dataset. The datasets are used to detection with GAN. arXiv preprint. 2018.
train the state-of-the-art transfer learning techniques to [18] Alireza M, Jonathon S, Navdeep J, Ian G, Brendan F. Adversarial
classify plant leaf diseases. The extensive simulation autoencoders. arXiv preprint. 2015
outcomes gives three important understandings to develop [19] Tim S, Ian G, Wojciech Z, Vicki C, Alec R, Xi C. Improved
a plant leaf disase detection model. First, the class size techniques for training GANs. arXiv preprint. 2016.
balanced augmented dataset gives a better classification [20] Yanghao L, Naiyan W, Jiaying L, Xiaodi H. Demistifying neural
accuracy than the original dataset. Second, the style transfer. arXiv preprint. 2017.
performance of deep learning based augmentation
[21] Khizar H. Super-resolution via deep learning. arXiv preprint. 2017
technique is higher than the manipulation techniques. At
last, the combination of multiple augmentation techniques [22] Dmitry U, Andrea V, Victor L. Instance normalization: the missing
ingredient for fast stylization. arXiv preprint. 2016.
based dataset gives a superior result than all other datasets.
In the future, additional image augmentation techniques [23] Philip TJ, Amir AA, Stephen B, Toby B, Boguslaw O. Style
such as autoencoders, Random erasing, Feature space augmentation: data augmentation via style randomiza-tion. arXiv e-
prints. 2018.
augmentation, Adversarial training, and Meta-learning
techniques can implement into the plant leaf disease [24] Josh T, Rachel F, Alex R, Jonas S, Wojciech Z, Pieter A. Domain
randomization for transferring deep neural networks from
dataset. simulation to the real world. arXiv preprint. 2017.
REFERENCES [25] Ashish S, Tomas P, Oncel T, Josh S, Wenda W, Russ W. Learning
[1] Geetharamani G, Arun Pandian J, Identification of plant leaf diseases from simulated and unsupervised images through adversarial
using a nine-layer deep convolutional neural network, Comput. training. In: Conference on computer vision and pattern recognition,
Electr. Eng. 76, 323–338, 2019. 2017

[2] Arun Pandian J, Geetharamani G, Mohit Agarwal, Suneet Kumar [26] Stephan RR, Vibhav V, Stefan R, Vladlen K. Playing for data:
Gupta, Plant Leaf Disease classification techniques using Leaf ground truth from computer games. In: European conference on
Images: A Study, International Journal of ChemTech Research, computer vision (ECCV ); 2016
Vol.12, No.05,pp212-216,2019. [27] Brostow Gabriel J, Fauqueur Julien, Cipolla Roberto. Semantic
[3] Arun Pandian J, G Geetharamani, identification of Fruit Categories object classes in video: a high-definition ground truth database.
using a nine-layer Deep Convolutional Neural Network, Vol.12, Pattern RecognLett. 2008;30(2):88–97.
No.05,pp238-246,2019

9th International Conference on Advanced Computing (IACC) 203

[28] Marius C, Mohamed O, Sebastian R, Timo R, Markus E, Rodrigo B,
Uwe F, Stefan R, Bernt S. The cityscape dataset for semantic urban
scene understanding. In: CVPR; 2016.
[29] Maayan F-A, Idit D, Eyal K, Michal A, Jacob G, Hayit G. GAN-
based synthetic medical image augmentation for increased CNN
performance in liver lesion classification. arXiv preprint. 2018.
[30] Changhee H, Hideaki H, Leonardo R, Ryosuke A, Wataru S,
Shinichi M, Yujiro F, Giancarlo M, Hideki N. GAN-based synthetic
brain mr image generation. In: 2018 IEEE 15th International
Symposium on biomedical imaging (ISBI 2018). IEEE, 2011. P.
734
[31] https://www.kaggle.com/arunpandianj/collective-augmented-plant-
leaf-disease-dataset
[32] https://www.kaggle.com/arunpandianj/manipulation-augmented-
plant-leaf-disease-dataset
[33] https://www.kaggle.com/arunpandianj/gan-augmented-plant-leaf-
disease-dataset
[34] J Arun Pandian, Gopal, Geetharamani, “Data for: Identification of
Plant Leaf Diseases Using a 9-layer Deep Convolutional Neural
Network”, Mendeley Data, v1, 2019
[35] https://www.kaggle.com/arunpandianj/nst-augmented-plant-leaf-
disease-dataset

204 9th International Conference on Advanced Computing (IACC)