Apple Fruits Categorizing Based On Deep Convolutional Neural Network Techniques
Apple Fruits Categorizing Based On Deep Convolutional Neural Network Techniques
Corresponding Author:
Nashaat M. Hussain Hassan
Department of Electronics and Electrical Communication Engineering, Faculty of Engineering
Fayoum University
Badr 63514, Egypt
Email: [email protected]
1. INTRODUCTION
The classification of fruits and vegetables [1]–[7] using computer vision techniques [8], [9], has
become one of the important research topics, due to the discrepancy in prices between different types. The
increased consumption of fruits and vegetables makes it necessary to increase the separation rate of the machine
per hour. On the other hand, climate changes have caused the emergence of many diseases in different
agricultural crops. Until now, the use of computer vision techniques in the classification of fruits still faces
many challenges, for many reasons, including the great similarity between types of fruits of the same type, the
need to train the technique on a large amount of data, and the quality and number of features suitable for
application. This is in addition to the need to always improve the efficiency of the technique and increase its
speed. All of these reasons led to the use of deep conventional neural network (DCNN) techniques [10]–[18]
becoming necessary. Since machine learning techniques [19]–[28] are still not suitable for applications in
which algorithm training is performed on a large amount of data, as well as applications in which a large
number of classes are separated. So, recently, fruits are recognised from images using deep neural networks
(DNN), which are utilised in the field of image identification and classification. Compared to other machine
learning methods, DNN performs better. Deep learning algorithms include convolutional neural networks
(CNNs), which are categorised as such. CNN [29] are the most popular type of artificial neural networks
(ANNs) used in deep learning [30]. In this paper, a proposed CNN model is presented. The proposed model is
used for the purpose of classifying 13 types of apples with high accuracy. The proposed CNN model is designed
to work by the best accuracy and with high processing speed. The proposed CNN method employed several
hidden layer and epoch combinations for various scenarios in order to compare their classification accuracy
results. The proposed method was applied to 64,040 images for model training, images belonging to 13 classes,
21,340 images belonging to 13 classes for modelvalidation, 21,340 images belonging to 13 classes for testing.
In this section, a number of recent publications are reviewed and analyzed, which dealt with presenting
proposals for techniques that contribute to the development of automatic separation of fruits and vegetables in
general, and apple fruits in particular based using CNN techniques.
Sakib et al. [29] method that is suggested uses deep learning to categorize five different types of
apples. Despite using a sizable database, the author assumed that the accuracy of the proposal would be
assessed using a variety of performance evaluation techniques. He also did not compare the accuracy of his
proposal's results to those of other proposals, the results of the processing speed were also not reviewed. This
is in addition to the fact that his proposal is to classify only five types of apple fruits.
Risdin et al. [31] proposed technique is to classify four different types of fruits: grape, green apple,
lemon, and lychee; based on the use of deep learning techniques. Although the author compared the results of
the proposed performance accuracy test with the results of the performance accuracy test of similar techniques,
and the proposed results are distinguished compared to the results of other techniques, it may be noted in this
research paper the following: First, he used a simple database of only 2,403 images distributed over the four
varieties. Secondly, fruits of the same type were not classified, but rather fruits of different types were
classified, and this is easier to classify. It was more beneficial to implement a classification of fruits of the same
type. Third, the author did not review the processing speed results of his proposed technique.
Yang and Cho [32] proposed technique is to classify seven different types of fruits: bell pepper,
strawberry, orange, lemon, pomegranate, pineapple, and banana; based on the use of deep learning algorithms.
Although the author used a large database, achieved high-performance accuracy, and reviewed the results of
processing speed, he did not classify fruits of the same kind, and as we mentioned before, classifying fruits of
the same type is much more difficult than classifying fruits of different types. The author did not compare the
results of testing the accuracy of his proposed technique with the results of other techniques. The proposed
model has succeeded in realizing a test accuracy of 98.9%. This paper is constructed as: in second section,
overview of the methodology of the proposed technique is presented. Results and discussion are demonstrated
in section three. In last section, conclusion and future lines are explained.
2. METHODOLOGY
The steps of the suggested method are included in this section. The suggested procedure starts with
database collection, followed by data pre-processing (feature scaling, dataset augmentation, dataset splitting),
CNN model construction, training, model validation, model testing, and hyper-parameter adjusting. Figure 1
shows the workflow of the CNN model we have suggested.
ReLU activation function, Adam optimizer,0.2 dropout rate, batch size at 16, epochs at 1, 6 layers, 3×3 CONV
kernal size, 2×2, max-pooling kernal size, 2 stride size, and 1 amount of zero padding.
show the ability of the proposed method to fully identify all of these types of apple fruits after the fifth epoch.
The results of performance accuracy, validation losses, and validating times are very similar to the training
results of the proposed method.
3.3. Confusion matrix results for testing the proposed convolutional neural network model
Two statistical performance metrics for classification tests are sensitivity and precision. The capacity
of the prediction model to choose an instance of a specific class from the dataset is referred to as sensitivity.
The percentage of genuine affirmative classifications that are accurately identified is what matters. Contrarily,
accuracy is defined as the percentage of accurately detected anticipated positive classes. They come from
(1) and (2).
𝑇𝑃
𝑠𝑒𝑛𝑠𝑖𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 = (𝑇𝑃+𝐹𝑁) (1)
𝑇𝑃
𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 = (𝑇𝑃+𝐹𝑃) (2)
The numbers for the true positive, false positive, and false negative forecasts for the class under
consideration are, respectively, true positive, false positive, and false negative. The results of the suggested
technique's confusion matrix (class sensitivity and class precision) for the thirteen different apple fruits are
shown in Table 5. The findings indicate that:
− Eight varieties could be distinguished with 100% accuracy.
− A whopping 99% of cultivars have been identified.
− A class that was discovered at a 95% rate, followed by another class that was discovered at a 92.6% rate.
− The ninth grade received an accuracy score of 86%, which was the lowest performance accuracy achieved
by the suggested technique.
− The proposed method's overall accuracy across the thirteen items was 98.9%.
Table 5. Confusion matrix for test accuracy of the proposed CNN model
Actual Predicted class Class
class Clas Clas Clas Clas Clas Clas Clas Clas Clas Clas Clas Clas Clas sensitivity
s1 s2 s3 s4 s5 s6 s7 s8 s9 s 10 s 11 s 12 s 13 %
Class 1 1520 120 0 0 0 0 0 0 0 0 0 0 0 92.6
Class 2 0 1480 0 0 0 0 0 0 0 0 0 0 0 100
Class 3 0 0 1600 0 0 0 0 0 0 0 0 0 0 100
Class 4 0 0 0 1640 0 0 0 0 0 0 0 0 0 100
Class 5 0 0 0 0 1540 70 0 0 0 0 0 0 0 95
Class 6 0 0 0 0 0 1640 0 0 0 0 0 0 0 100
Class 7 0 0 0 0 0 0 1520 0 0 0 0 0 0 100
Class 8 0 0 0 0 0 0 0 1630 0 1 0 0 0 99
Class 9 0 0 0 0 0 0 260 0 1390 0 0 0 0 86
Class 10 0 0 0 0 0 0 0 1 0 1430 0 0 0 99
Class 11 0 0 0 0 0 0 0 0 0 0 1660 0 0 100
Class 12 0 0 0 0 0 0 0 0 0 0 0 1640 0 100
Class 13 0 0 0 0 0 0 0 0 0 0 0 0 2190 100
Class 100 92.5 100 1 100 96 8 99 100 99 100 100 100 Overall
precisio 0 5. correctness
n% 0 5 = 98.9
Apple fruits categorizing based on deep convolutional neural network … (Nashaat M. Hussain Hassan)
3700 ISSN: 2252-8938
3.4. Comparesion between the final test results of the suggested model and the related models
In this section, a comparison is presented between the results of testing the proposed classifier and the
results of testing a group of transfer learning classifiers. The results of testing the proposed technique for
identifying 13 types of apple fruits appear in Table 6. These results include test accuracy, recall and, f1-score,
support, batch size, and epoch number. The results show the high accuracy of the proposed technique in
identifying different types of apple fruits as best as possible, as the average accuracy of the proposed model in
test for identifying thirteen different types of apple fruits reached 98.9%.
Table 6. Final test accuracy, recall, fa-score, support, batch size, epotch number of the proposed classifier
Epoch number Batch size Test accuracy (%) Recall F1-score Support
10 16 98.9 1 1 166
5 16 98.9 1 1 166
On the other hand, Table 7 displays the test results of four of the best classifiers, which are VGG16,
EfficientNetV2M, MobileNetV2, and InseptionV3 to identify only eight different types of apple fruits as
mentioned in the research paper referred to in Cortés et al. [34]. These results include training time, test
accuracy, recall, f1-score, and support, batch size, epoch number. The results show that the VGG16 classifier
achieved an accuracy of 69.89%, the EfficientNetV2M classifier achieved an accuracy of 70.54%, the
MobileNetV2 classifier achieved an accuracy of 91%, and finally the InseptionV3 classifier achieved an
accuracy of 92.96%. The results also show that this accuracy was achieved after a number of epochs amounting
to 100 epochs. The results also show that training times varied between one and four hours. Therefore, all of
these results show the significant superiority in favor of the proposed classifier over the other classifiers in all
directions, whether in the number of classes 13 versus 8, the performance accuracy is 98.9% in favor of the
proposed versus 69.89% to 92.96%, training times 7.18 seconds versus between 1.23 hours to 4.02 hours, and
finally epochs conts, as the proposal requires 3 to 5 epochs to fully identify the different types. In contrast,
other techniques exceeded 100 epochs to achieve the obtained accuracy.
Table 7. Test accuracy, recall, f1-score, support, batch size, epotch number, and training time for different
related algorithms
CNN architecture Test accuracy (%) Batch size Epoch number Recall F1-score Support Training time
VGG 16 69.89 24 100 0.589 0.688 49 1:39h
EfficientNetV2M 70.54 24 100 0.725 0.675 46 4:02h
MobileNetV2 91 24 100 0.734 0.890 62 1:23h
InseptionV3 92.96 24 100 0.978 0.929 67 1:27h
4. CONCLUSION
Design and implementation of apple fruits classification system based on CNN algorithm is presented
in this work. The designed model works to classify thirteen types of apple fruits with high accuracy and high
processing speed. The proposed technique was based on training and testing the model on a maximum number
of images of apple fruits, by increasing the number of database images tenfold, after augmentation was
performed on the images. The technology also relied on good tuning of the hyperparameters. To further ensure
the efficiency of training, validation was performed on 20% of the database. All results that demonstrate the
high efficiency of the proposed model were reviewed. The results of the proposal were compared with the
results of four related techniques. The results showed the great advantage of the proposed technology at all
levels. In the proposed method 64040 images for model training were used, images belonging to 13 classes and
21340 images belonging to 13 classes for model validating and testing. The proposed model has succeeded in
realizing a test accuracy of 98.9%. Future lines of this work will focus on the following: first, modify the
proposed model to classify the type of apple, the apple fruit helthy or defected, and then the defect type for
each one. Second, think about the hardware implementation of the technique that gives the best results (high
accuracy in a very low time), so that it can be used in industrial enterprises that rely on computer vision
techniques.
REFERENCES
[1] F. Femling, A. Olsson, and F. A. -Fernandez, “Fruit and vegetable identification using machine learning for retail applications,” in
2018 14th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS), Nov. 2018, pp. 9–15, doi:
10.1109/SITIS.2018.00013.
[2] A. A. Nashat and N. M. H. Hassan, “Automatic segmentation and classification of olive fruits batches based on discrete wavelet
transform and visual perceptual texture features,” International Journal of Wavelets, Multiresolution and Information Processing,
vol. 16, no. 1, Jan. 2018, doi: 10.1142/S0219691318500030.
[3] N. M. H. Hassan and A. A. Nashat, “New effective techniques for automatic detection and classification of external olive fruits
defects based on image processing techniques,” Multidimensional Systems and Signal Processing, vol. 30, no. 2, pp. 571–589, Apr.
2019, doi: 10.1007/s11045-018-0573-5.
[4] K. Tan, W. S. Lee, H. Gan, and S. Wang, “Recognising blueberry fruit of different maturity using histogram oriented gradients and
colour features in outdoor scenes,” Biosystems Engineering, vol. 176, pp. 59–72, Dec. 2018, doi:
10.1016/j.biosystemseng.2018.08.011.
[5] S. Sabzi, Y. A. -Gilandeh, and G. G. -Mateos, “A new approach for visual identification of orange varieties using neural networks
and metaheuristic algorithms,” Information Processing in Agriculture, vol. 5, no. 1, pp. 162–172, Mar. 2018, doi:
10.1016/j.inpa.2017.09.002.
[6] A. Pande, M. Munot, R. Sreeemathy, and R. V. Bakare, “An efficient approach to fruit classification and grading using deep
convolutional neural networ,” in 2019 IEEE 5th International Conference for Convergence in Technology (I2CT), Mar. 2019, pp.
1–7, doi: 10.1109/I2CT45611.2019.9033957.
[7] M. S. Hossain, M. Al-Hammadi, and G. Muhammad, “Automatic fruit classification using deep learning for industrial applications,”
IEEE Transactions on Industrial Informatics, vol. 15, no. 2, pp. 1027–1034, Feb. 2019, doi: 10.1109/TII.2018.2875149.
[8] S. L. Raja, N. Ambika, V. Divya, and T. Kowsalya, “Fruit classification system using computer vision: a review,” International
Journal of Trend in Research and Development, vol. 5, no. 1, pp. 22–26, 2018.
[9] H. Mureşan and M. Oltean, “Fruit recognition from images using deep learning,” Acta Universitatis Sapientiae, Informatica, The
Journal of Sapientia Hungarian University of Transylvania, vol. 10, no. 1, pp. 26–42, Dec. 2018.
[10] N. M. H. Hassan, B. R. G. Elshoky, and A. M. M. Mabrouk, “Quality of performance evaluation of ten machine learning algorithms
in classifying thirteen types of apple fruits,” Indonesian Journal of Electrical Engineering and Computer Science, vol. 30, no. 1,
pp. 102–109, Apr. 2023, doi: 10.11591/ijeecs.v30.i1.pp102-109.
[11] Z. M. Khaing, Y. Naung, and P. H. Htut, “Development of control system for fruit classification based on convolutional neural
network,” in 2018 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus), Jan.
2018, pp. 1805–1807, doi: 10.1109/EIConRus.2018.8317456.
[12] A. Nasiri, A. T. -Garavand, and Y.-D. Zhang, “Image-based deep learning automated sorting of date fruit,” Postharvest Biology
and Technology, vol. 153, no. 3, pp. 133–141, Jul. 2019, doi: 10.1016/j.postharvbio.2019.04.003.
[13] N. Ismail and O. A. Malik, “Real-time visual inspection system for grading fruits using computer vision and deep learning
techniques,” Information Processing in Agriculture, vol. 9, no. 1, pp. 24–37, Mar. 2022, doi: 10.1016/j.inpa.2021.01.005.
[14] S. Naik and B. Patel, “Machine vision based fruit classification and grading - A review,” International Journal of Computer
Applications, vol. 170, no. 9, pp. 22–34, Jul. 2017, doi: 10.5120/ijca2017914937.
[15] P. Moallem, A. Serajoddin, and H. Pourghassem, “Computer vision-based apple grading for golden delicious apples based on
surface features,” Information Processing in Agriculture, vol. 4, no. 1, pp. 33–40, Mar. 2017, doi: 10.1016/j.inpa.2016.10.003.
[16] J. L. R. -Aranda, J. I. N. -Varela, J. C. C. -Tello, and G. R. -Ramirez, “Fruit classification for retail stores using deep learning,” in
MCPR 2020: Pattern Recognition, 2020, pp. 3–13, doi: 10.1007/978-3-030-49076-8_1.
[17] A. Kader, S. Sharif, P. Bhowmick, F. H. Mim, and A. Y. Srizon, “Effective workflow for high-performance recognition of fruits
using machine learning approaches,” International Research Journal of Engineering and Technology (IRJET), vol. 7, no. 2, pp.
1516–1521, 2020.
[18] L. S. P. Annabel, T. Annapoorani, and P. Deepalakshmi, “Machine learning for plant leaf disease detection and classification – A
review,” in 2019 International Conference on Communication and Signal Processing (ICCSP), Apr. 2019, pp. 0538–0542, doi:
10.1109/ICCSP.2019.8698004.
[19] A. Nosseir and S. E. A. Ahmed, “Automatic classification for fruits’ types and identification of rotten ones using k-NN and SVM,”
International Journal of Online and Biomedical Engineering (iJOE), vol. 15, no. 3, pp. 47–61, Feb. 2019, doi:
10.3991/ijoe.v15i03.9832.
[20] H. M. W. M. Hippola, D. P. W. Mesthri, R. M. T. P. Rajakaruna, L. Yasakethu, and M. Rajapaksha, “Machine learning based
classification of ripening and decay stages of mango (Mangifera indica L.) cv. Tom EJC,” in 2022 2nd International Conference
on Image Processing and Robotics (ICIPRob), Mar. 2022, pp. 1–6, doi: 10.1109/ICIPRob54042.2022.9798722.
[21] D. G. Koç and M. Vatandaş, “Classification of some fruits using image processing and machine learning,” Turkish Journal of
Agriculture - Food Science and Technology, vol. 9, no. 12, pp. 2189–2196, Dec. 2021, doi: 10.24925/turjaf.v9i12.2189-2196.4445.
[22] A. A. A. El-aziz, A. Darwish, D. Oliva, and A. E. Hassanien, “Machine learning for apple fruit diseases classification system,” in
AICV 2020: Proceedings of the International Conference on Artificial Intelligence and Computer Vision (AICV2020), 2020, pp.
16–25, doi: 10.1007/978-3-030-44289-7_2.
[23] K. Sabanci and M. F. Unlersen, “Different apple varieties classification using kNN and MLP algorithms,” International Journal of
Intelligent Systems and Applications in Engineering, vol. 4, pp. 166–169, 2016.
[24] N. Chaithra, B. Madhu, K. Umamaheswari, and S. Balasubramanian, “Classification and regression tree (CART) algorithm for the
prediction of ischemic heart disease,” International Journal of Future Generation Communication and Networking, vol. 13, no. 4,
pp. 196–206, 2020.
[25] A. Tharwat, T. Gaber, A. Ibrahim, and A. E. Hassanien, “Linear discriminant analysis: A detailed tutorial,” AI Communications,
vol. 30, no. 2, pp. 169–190, May 2017, doi: 10.3233/AIC-170729.
[26] S. Wang and Q. Chen, “The study of multiple classes boosting classification method based on local similarity,” Algorithms, vol. 14,
no. 2, Jan. 2021, doi: 10.3390/a14020037.
[27] D. A. McCarty, H. W. Kim, and H. K. Lee, “Evaluation of light gradient boosted machine learning technique in large scale land use
and land cover classification,” Environments, vol. 7, no. 10, Oct. 2020, doi: 10.3390/environments7100084.
[28] F. R. Lumbanraja, E. Fitri, Ardiansyah, A. Junaidi, and R. Prabowo, “Abstract classification using support vector machine algorithm
(case study: Abstract in a computer science journal),” Journal of Physics: Conference Series, vol. 1751, no. 1, Jan. 2021, doi:
10.1088/1742-6596/1751/1/012042.
[29] S. Sakib, Z. Ashrafi, and M. A. B. Siddique, “Implementation of fruits recognition classifier using convolutional neural network
algorithm for observation of accuracies for various hidden layers,” ArXiv e-Journal, pp. 1-4, Apr. 2019.
[30] F. M. A. Mazen and A. A. Nashat, “Ripeness classification of bananas using an artificial neural network,” Arabian Journal for
Science and Engineering, vol. 44, no. 8, pp. 6901–6910, Aug. 2019, doi: 10.1007/s13369-018-03695-5.
[31] F. Risdin, P. K. Mondal, and K. M. Hassan, “Convolutional neural networks (CNN) for detecting fruit information using machine
learning techniques,” IOSR Journal of Computer Engineering (IOSR-JCE), vol. 22, no. 2, pp. 01–13, 2020.
Apple fruits categorizing based on deep convolutional neural network … (Nashaat M. Hussain Hassan)
3702 ISSN: 2252-8938
[32] M. Yang and S. I. Cho, “Fruit classification using convolutional neural network (CNN),” Precision Agriculture Science and
Technology, vol. 3, no. 1, pp. 1–8, 2021, doi: 10.12972/pastj.20210001.
[33] M. Oltean, “Fruits-360 dataset,” Kaggle, 2021. [Online]. Available: https://www.kaggle.com/datasets/moltean/fruits
[34] S. G. Cortés, A. M. Díaz, J. A. O. Prendes, and A. B.García, “Transfer learning with convolutional neural networks for cider apple
varieties classification,” Agronomy, vol. 12, no. 11, Nov. 2022, doi: 10.3390/agronomy12112856.
BIOGRAPHIES OF AUTHORS
Nashaat M. Hussain Hassan was born in Quena, Egypt, in 1977. He received his
B.Sc. in communication and electronics engineering from Al-Azhar University – Egypt in 2002.
In 2005, he received his M.Sc. degree in communication and electronics engineering from
(C.N.M.) National Center of Microelectronics, Seville University – Spain. In 2009, he received
his Ph.D. in Digital Integrated Circuit Design for the Applications of Image processing from
(C.N.M.) National Center of Microelectronics, Seville University – Spain. In October 2019 he
was promoted to the position of an Associate Professor position. Currently, he is working as an
associate professor in the Department of Electronics & Electrical communication, Faculty of
Engineering, Fayoum University, Egypt. His research interest includes algorithms development
(analysis, design and improvement) and full-cycle software & hardware product development
(Matlab, C, C++, VHDL, FPGA, and Xilinx). He can be contacted at email:
[email protected].
Gihan H. Zaki received the B.Sc. degree in Physics from Faculty of Science, Cairo
University in in 1988 and the M.Sc. in Experimental Solid State Physics from Cairo University
in1995. In 2011, he received his Ph.D. in Theoretical Nano-Technology Physics in Faculty of
Science, Beni-Suef University. From 1995 till 2012, he was working as a lecturer assistant and
then as a lecturer at 6 October university-Egypt. From 2014 until now, he is working as a lecturer
in Giza High Institute of Engineering and Technology. He can be contacted at email:
[email protected].