0% found this document useful (0 votes)
68 views

Damage Identification of Selected Car Parts Using Image Classification and Deep Learning

This document summarizes a study that uses image classification and deep learning to identify damage to selected car parts for automotive insurance claims processing. A two-level machine learning model was developed to first classify images of car parts (front bumper, rear bumper, wheel) and then detect the presence of any damage. The model was trained on images from Google and achieved 94.84% accuracy for part classification and 97.16% accuracy for damage identification. The goal was to help automate the insurance claims process by analyzing images of vehicle damage uploaded by customers.
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
68 views

Damage Identification of Selected Car Parts Using Image Classification and Deep Learning

This document summarizes a study that uses image classification and deep learning to identify damage to selected car parts for automotive insurance claims processing. A two-level machine learning model was developed to first classify images of car parts (front bumper, rear bumper, wheel) and then detect the presence of any damage. The model was trained on images from Google and achieved 94.84% accuracy for part classification and 97.16% accuracy for damage identification. The goal was to help automate the insurance claims process by analyzing images of vehicle damage uploaded by customers.
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 5

Damage Identification of Selected Car Parts Using

Image Classification and Deep Learning

Abraham C. Chua, Christian Rei B. Mercado, John Phillip R. Pin, Angelo Kyle T. Tan,
2021 IEEE 13th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment, and Management (HNICEM) | 978-1-6654-0167-8/21/$31.00 ©2021 IEEE | DOI: 10.1109/HNICEM54116.2021.9731806

Jose Benito L. Tinhay, Elmer P. Dadios, and Robert Kerwin C. Billones

Department of Manufacturing Engineering and Management


De La Salle University, Manila, Philippines

E-mail: {abraham_c_chua, christian_mercado, john_phillip_pin, angelo_kyle_tan, jose_benito_tinhay,


elmer.dadios, robert.billones}@dlsu.edu.ph

Abstract — This study presents the use of image classification and convolutional neural network. Three common car parts were
deep learning in the field of insurance claims and management for considered which includes front bumper, rear bumper, and car
the identification and assessment of damaged vehicle parts. wheels machine. The image dataset was taken from Google
Vehicular insurance claims on require appraisers to decide the images.
damage of the vehicles. A two-level machine learning-based system
was developed to classify different car parts (front bumper, rear
II. REVIEW OF RELATED LITERATURE
bumper, and car wheels), and to detect the presence of any
damages. The image dataset used in the study was obtained from
a Google image. This dataset is used for training and validation of
A. Machine Learning in Automotive Insurance Process and
the convolutional neural network (CNN) model. The first model Management
yields a training accuracy of 94.84% and validation accuracy of
81.25% for car parts classification. The second model yields a The have been demands to find ways to speed up the auto
training accuracy of 97.16% and validation accuracy of 49.28% insurance process. One of the solutions that was proposed is by
for damage identification. prompting customers to upload pictures of their vehicle’s
damages using their mobile phones. The pictures can be viewed
Keywords — convolutional neural networks, damage evaluation, by insurance agents to investigate the damages and process
image classification, insurance claims, machine learning
their insurance claims as fast as possible. This system helped in
speeding up the car insurance claim process, but some people
I. INTRODUCTION discovered a flaw in the system and take advantage of the
situation by submitting multiple copies of a supposedly single
Automobile insurance processes typically require the
insurance case which results in huge losses to insurance
performance and verdict of an appraiser. These appraisers must
companies [2]. The application and integration of both machine
be objective with regards to how they determine the amount of
learning and machine vision to the automotive insurance
damage that can be noticed in the externals of the vehicle to
industry can greatly decrease the insurance leakage caused by
assess the cost to repair the certain part damaged. According to
human error and fraud. In one study, the Moiré effect detection
data presented by the Beijing Bureau of China Insurance
was used to detect and classify images to determine whether the
Regulatory Commission, about 10 percent of all insurance
picture was downloaded from the internet to help safeguard the
claims for vehicles are insurance frauds one way or another
insurance company against individuals who may try to cheat the
which are massive threats not just to the insurance company,
system [3].
but also to the entire industry [1]. To reduce the amount of
human error present in the processes done in performing
vehicle insurance-related occupations, this study developed a The Mask Region-Based Convolutional Neural Networks
machine learning model which can detect and classify the car (R-CNN), PANet, and a combination of the two with VGG16
parts and the presence of any damages. This process can fast network was used to detect and classify the different car parts
track the damage appraisal and eliminate the possibility of bias [4]. In other studies, feature representation techniques were
during the assessment. The machine learning model uses image used for pattern recognition. This processing techniques use
classification which allows it to study the relationship of pixels raw data to discover patterns needed for detection [5]. Other
in the image so that it can find distinct similarities and techniques that used Move Learning, and Outfit Learning
differences which can be used for predicting a certain outcome. exhibited a significant result in the issue of vehicle harm
In this study, the machine learning model used was a characterization [6]. The utilization of different types of deep
learning concepts were also helpful in the advancement of

978-1-6654-0167-8/21/$31.00 ©2021 IEEE

Authorized licensed use limited to: Pusan National University Library. Downloaded on September 12,2023 at 13:40:50 UTC from IEEE Xplore. Restrictions apply.
image-based detection, classification, and recognition The data that was used in the dataset was collected from Google
problems. One example is the Convolutional Neural Networks images.
(CNN) which can be used for domain-specific pre-training and
fine-tuning [7][8][9]. The use of Auto-encoders and other
unsupervised pre-training techniques on datasets with small
datasets have been shown to increase the performance of the
classifier. While most of the supervised pre-training techniques
require bigger datasets. A combination of transfer and ensemble
training would yield a more accurate deep learning code.

B. Object Detection and Classification in Vehicle Detection


Systems Fig. 1. Left to Right: Undamaged Front Bumper, Rear Bumper, and Car
Wheel
Segmentation techniques worked well in the assessment
program for cars that have been damaged and covered by
insurance. Using this technique, a system can be created
without using manpower to check the physical conditions of a
car. A system with object and video detection capability
coupled with a user-interface can send results within seconds
[10][11]. Convolutional Neural networks were also used to
verify images of car accidents whether they had common or
severe damage. In one experimental setup, a system had 5 Fig. 2. Left to Right: Damaged Front Bumper, Rear Bumper, and Car Wheel
continuous modules. Each model was given an attribute to B. Data Augmentation
judge a given image. This study used web pictures of real car
damages. As a result, it has an 88% theoretical accuracy, and The training dataset involves both damaged and
has the same results in the real world [12]. undamaged car parts. The second model involves the use of six
The automotive industry is having a hard time detecting classifications, where each classification corresponds to the
defects such as dings and dents on car body surfaces. These type of car parts and presence of any damages (damage or no
anomalies can be solved by using optical flow algorithms and damage). The dataset and classification for the second model
the deflectometry principle. The methods and processes can be include Front Bumper, Rear Bumper, Car Wheels, Damaged
completed within 15 seconds. The method uses light patterns to Front Bumper, Damaged Rear Bumper, and Damaged Car
check the area, with this fusion law and optical flow are Wheels. Due to the limited nature of the data collection method,
integrated to obtain a fused image of all defects or anomalies. the model development could only utilize a small dataset. For
To check if the fused image is correct, a new post-processing the classification experiments, the datasets are randomly split,
technique was applied. After cross-checking, the image where 60%-80% was used for training and 20-40% was used
underwent a blurring method to review each pixel. The result is for testing. Table I and Table II describes the size of our train
obtained by performing subtraction law where the defects are and test sets. There are a total of 30 test images for the car parts
differentiated from other surface deformations [13]. Defect classification, where 10 images were used for each class. Table
detection can be done by various methods, but not adequate for I describes the dataset utilized for the car parts classification.
detecting multiple scales. Using Variance Of Variance (VOV) There are a total of 30 test images for damage classification,
method and integral image method, it is possible to detect where 5 images were used for each class. Table II describes the
defects with intensive irregularity [14]. The method uses dataset utilized for the damage classification.
captured images and turns it into multiple parts or patches to be
analyzed. Detecting the anomalies of images with low
resolution, and uneven illumination is still possible by just
changing the size of the patch. Other object detection methods Table I. Description of Data Set for parts classification
used fuzzy logic controllers [15] for image processing as a
system, remote sensing [16], and image pyramid networks [17]. Classes Train Size Validation Test

III. METHODOLOGY Front Bumper 161 53 10

A. Dataset Description Rear Bumper 206 91 10

The image dataset was collected through a local Car Wheel 161 53 10
automotive dealer. The car parts classification will include
three parts, namely front bumper, car wheel, and rear bumper.

Authorized licensed use limited to: Pusan National University Library. Downloaded on September 12,2023 at 13:40:50 UTC from IEEE Xplore. Restrictions apply.
Table II. Description of Data Set for damage classification models. The first model is tasked in identifying which part of
the car is being shown. The first model used dataset 1 which
Classes Train Size Validation Test includes 3 classifications. CNN Model 1 is the first level
detection of the system. The second model is tasked in
Undamaged Front
Bumper
179 95 5 determining whether the specific car part is damaged or not
damaged. The second model used dataset 2 which includes 6
Undamaged Rear
206 43 5
classifications. CNN Model 2 is the second level detection of
Bumper the system.
Undamaged Car
224 118 5 E. Performance Evaluation
Wheel

Damaged Front The performance metrics evaluation involved the


177 63 5
Bumper calculations for accuracy, precision, recall, and f1-score of the
models.
Damaged Rear
110 47 5
Bumper

Damaged Car = (1)


202 65 5
Wheel

! "
= (2)
! " #$ ! "
C. Training a CNN
Training the CNN started with random initialization. For
both datasets, the CNN model building consisted of 6 layers: % && =
! "
(3)
Convolution2 - Pooling2 - Convolution2 - Pooling2 - ! " #$ ' "
Convolution2 - Pooling2 - Convolution2 - Pooling2 - RELU -
Softmax. The first two convolution layers have 32 channels

with size of 3 x 3. All convolutional layers have a size of 3 x 3, (1 − =2∙ (4)
#
32 channels for the first two, then 64 channels for the
remaining. Each pooling layer has a pool size of 2 x 2. Dataset III. RESULTS AND DISCUSSION
for parts classification has parameters of 75 epochs, a batch size
of 16, img_width, img_height = 175, 175 and activation The CNN model training, validation, and testing results are
functions softmax and ReLU (Rectified Linear Unit). The presented here. It includes the performance metrics associated
dataset for damage classification has parameters of 150 epochs, with the generated CNN models. Table III shows the values of
a batch size of 32, img_width, img_height = 75, 75. The dataset different metrics for car parts dataset classification. The model
scale was set to 1x with 255 degrees rotation, shear-range of yields a training accuracy of 94.84% using 585 images, and
0.1, zoom-range of 0.1, and with horizontal-flip validation accuracy of 81.25% using 197 images. From these
results, the performance of the model is enough to be used on
D. Experimental Setup
prediction. Prediction accuracy for rear bumper is 60%, car
wheel is 90%, and front bumper is 80%. The model yields 100%
for precision, recall, and F1-score metrics in all classes. The
60% accuracy for rear bumper is mainly due to the images of
rear bumper and front bumper looking similar.

Table III. Classification performance for car parts

Fig. 3. Experimental Flowchart

Figure 3 shows the block diagram for model development.


There are two levels which involved the training of two CNN

Authorized licensed use limited to: Pusan National University Library. Downloaded on September 12,2023 at 13:40:50 UTC from IEEE Xplore. Restrictions apply.
V. CONCLUSION

The study applied Image classification and deep learning-


based algorithms for identifying and assessing damaged vehicle
car parts. It implemented a two-level classification; first level -
classification of cart parts (1) front bumper, (2) rear bumper,
(3) car wheel; and second level - damage classification (1)
Undamaged Front Bumper, (2) Undamaged Rear Bumper, (3)
Undamaged Car Wheel, (4) Damaged Front Bumper, (5)
Damaged Rear Bumper, (6) Damaged Car Wheel. The images
were collected manually from the internet. CNN models that
consisted of 6 layers were trained and tested. The study
Fig. 4. Training and validation accuracy plot of successfully obtained satisfactory results in model performance
car parts classification
which were measured using the models’ accuracy, precision,
recall, and F1-score. It can be observed that there are some
Table IV shows the values of different metrics for the damage
misclassifications when the CNN models were tested. It may be
classification dataset. The researchers used the same test dataset
due to the limited number of images in the dataset used in the
in this classification. Using 150 epochs, a batch size of 32, and
study. Moreover, a limited amount of damaged car part images
activation functions softmax and ReLU (Rectified Linear Unit)
from the web with some images having a low resolution may
Activation Function, the model obtained a training accuracy of
contributed to the misclassifications. It is recommended to have
97.16%, while the validation accuracy is 49.28%. It was
a larger datasets of car parts and vehicle damages. Image
observed that the performance of the with a small validation
resolution should also be considered in the data collection and
dataset is not sufficient to get a high accuracy. The model yields
filtering. Lastly, other machine learning models should also be
a prediction accuracy of 100% for Undamaged Front Bumper,
explored to improve the performance of the models.
Undamaged Rear Bumper, Undamaged Car Wheel, and
Damaged Car Wheel. Meanwhile, a prediction accuracy of 40%
for Damaged Front Bumper, and 20% for Damaged Rear ACKNOWLEDGMENTS
Bumper were achieved. The model yields 100% for precision,
The authors highly appreciate the Office of the Vice
recall, and F1-score metrics in all classes, except for
Chancellor for Research and Innovation (VCRI) of De La Salle
Undamaged Car Wheel.
University, Manila for the conference publication support, and
Table IV. Classification performance for damaged and undamaged car parts
Gokongwei College of Engineering (GCOE) for providing an
avenue for the researchers in the conduct of this research study

REFERENCES

[1] X. Dongwei, "Irrationality in automobile insurance classification:


Analysis of one problem in China automobile insurance market,"
2011 IEEE 3rd International Conference on Communication
Software and Networks, Xi'an, 2011, pp. 86-88, doi:
10.1109/ICCSN.2011.6014224.

[2] P. Li, B. Shen, W. Dong. “An Anti-fraud System for Car Insurance
Claim Based on Visual Evidence“ Computer Science and Pattern
Recognition. pp. 1-6. Apr. 2018. Doi: arXiv:1804.11207

[3] U. Waqas, N. Akram, S. Kim, D. Lee and J. Jeon, "Vehicle Damage


Classification and Fraudulent Image Detection Including Moiré
Effect Using Deep Learning," 2020 IEEE Canadian Conference on
Electrical and Computer Engineering (CCECE), London, ON,
Canada, pp. 1-5, 2020. doi: 10.1109/CCECE47787.2020.9255806.

[4] Y. Bengio, A. Courville and P. Vincent, "Representation Learning:


A Review and New Perspectives," in IEEE Transactions on Pattern
Analysis and Machine Intelligence, vol. 35, no. 8, pp. 1798-1828,
Aug. 2013, doi: 10.1109/TPAMI.2013.50.

[5] R. K. C. Billones et al., "Vision-Based Passenger Activity Analysis


Fig. 5. Training and validation accuracy plots of System in Public Transport and Bus Stop Areas," 2018 IEEE 10th
damage classification International Conference on Humanoid, Nanotechnology,

Authorized licensed use limited to: Pusan National University Library. Downloaded on September 12,2023 at 13:40:50 UTC from IEEE Xplore. Restrictions apply.
Information Technology,Communication and Control, 2011 International Symposium on Image and Data Fusion,
Environment and Management (HNICEM), 2018, pp. 1-6, doi: Tengchong, Yunnan, pp. 1-4, 2011. doi:
10.1109/HNICEM.2018.8666357. 10.1109/ISIDF.2011.6024246.

[6] P. Sarath, M. Soorya, S. A. Rahman, A., K. S. Suresh, & K. Devaki, [17] A. K. Brillantes et al., "Philippine License Plate Detection and
“Assessing Car Damage using Mask R-CNN.” International Classification using Faster R-CNN and Feature Pyramid Network,"
Journal of Engineering and Advanced Technology (IJEAT), 9, pp. 2019 IEEE 11th International Conference on Humanoid,
1-4, Feb.2020. doi:10.35940/ijeat.C5302.029320 Nanotechnology, Information Technology, Communication and
Control, Environment, and Management ( HNICEM ), 2019, pp. 1-
[7] K. Patil, M. Kulkarni, A. Sriraman and S. Karande, "Deep Learning 5, doi: 10.1109/HNICEM48295.2019.9072754.
Based Car Damage Classification," 2017 16th IEEE International
Conference on Machine Learning and Applications (ICMLA),
Cancun, pp. 50-54,2017. doi: 10.1109/ICMLA.2017.0-179.

[8] R. K. C. Billones et al., "Vehicle-Pedestrian Classification with


Road Context Recognition Using Convolutional Neural Networks,"
2018 IEEE 10th International Conference on Humanoid,
Nanotechnology, Information Technology,Communication and
Control, Environment and Management (HNICEM), 2018, pp. 1-6,
doi: 10.1109/HNICEM.2018.8666257.

[9] Billones R.K.C., Bandala A.A., Gan Lim L.A., Sybingco E., Fillone
A.M., Dadios E.P. (2020) Visual Percepts Quality Recognition
Using Convolutional Neural Networks. In: Arai K., Kapoor
S. (eds) Advances in Computer Vision. CVC 2019. Advances in
Intelligent Systems and Computing, vol 944. Springer, Cham.
https://doi-org.dlsu.idm.oclc.org/10.1007/978-3-030-17798-0_52

[10] W. Zhang, Y. Cheng, X. Guo, Q. Guo, J. Wang, Q. Wang, C. Jiang,


M. Wang, F. Xu & W. Chu Automatic Car Damage Assessment
System: Reading and Understanding Videos as Professional
Insurance Inspectors. Proceedings of the AAAI Conference on
Artificial Intelligence, 34(09), 13646-13647. 2020
https://doi.org/10.1609/aaai.v34i09.7110

[11] M. Guillermo et al., "Detection and Classification of Public


Security Threats in the Philippines Using Neural Networks," 2020
IEEE 2nd Global Conference on Life Sciences and
Technologies(LifeTech), 2020, pp. 320-324, doi:
10.1109/LifeTech48969.2020.1570619075

[12] A. Sharma, A. Verma, and D. Gupta, “Preventing Car Damage


using CNN and Computer Vision,” International Journal of
Innovative Technology and Exploring Engineering (IJITEE), 9, pp.
1-5, Nov-2019. [Online]. Available: https://www.ijitee.org/wp-
content/uploads/papers/v9i1/A5020119119.pdf.

[13] L. Arnal, J. J. Molina, E. Solanes & J. Tornero. “Detecting dings


and dents on specular car body surfaces based on optical flow.”
Journal of Manufacturing Systems, 45, pp.306-321. 2017.
doi:10.1016/j.jmsy.2017.07.006.

[14] B. Kwon, D. Kang. "A Multi-scale Probabilistic Algorithm that can


find Surface Defects on the Car Side-Protector." World Academy
of Science, Engineering and Technology, 46, pp. 1-6, 2009.
[online] Available:
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.308.160
4&rep=rep1&type=pdf

[15] R. K. C. Billones et al., "Vision-Based Passenger Activity Analysis


System in Public Transport and Bus Stop Areas," 2018 IEEE 10th
Internationa Conference on Humanoid, Nanotechnology,
Information Technology,Communication and Control,
Environment and Management (HNICEM), 2018, pp. 1-6, doi:
10.1109/HNICEM.2018.8666357.

[16] Y. Fang, J. Chen, Y. Xia, W. Song and Y. Yang, "Research on


Adaptive Classification Algorithm of Remote Sensing Image,"

Authorized licensed use limited to: Pusan National University Library. Downloaded on September 12,2023 at 13:40:50 UTC from IEEE Xplore. Restrictions apply.

You might also like