Machine Learning with Applications 9 (2022) 100371

Contents lists available at ScienceDirect

Machine Learning with Applications

journal homepage: www.elsevier.com/locate/mlwa

A deep convolutional neural network-based approach for detecting burn

severity from skin burn images
Sayma Alam Suha, Tahsina Farah Sanam ∗
Institute of Appropriate Technology, Bangladesh University of Engineering and Technology (BUET), Dhaka, Bangladesh

ARTICLE INFO ABSTRACT

Keywords: Burns being one of the leading causes of clinically significant morbidity can lead to a dramatic physiological
Burn severity reaction with prolonged repercussions, metabolic disturbance, severe scarring, catastrophic organ failure, and
Deep convolutional neural network death if not properly treated. Appropriate burn treatment management is associated with the severity of burn
Transfer learning
wounds which can be extremely challenging to anticipate at an early stage due to various factors using
Feature extraction
traditional clinical methods. Therefore, this study proposed a Deep Convolutional Neural Network (DCNN)
Image classification
based approach for detecting the severity of burn injury utilizing real-time images of skin burns. The DCNN
architecture leverage the utilization of transfer learning with fine tuning employing three types of pretrained
models on top of multiple convolutional layers with hyperparameter tuning for feature extraction from the
images and then a fully connected feed forward neural network to classify the images into three categories
according to their burn severity : first, second and third degree burns. In order to validate the efficacy of
the suggested strategy, the study also applies a traditional solution to mitigate this multi-class categorization
problem, incorporating rigorous digital image processing steps with several conventional machine learning
classifiers and then conducts a comparative performance assessment. The study’s findings demonstrate that
using pretrained models, the recommended DCNN model has gained significantly greater accuracy, with the
highest accuracy being obtained using the VGG16 pretrained model for transfer learning with an accuracy
of 95.63% . Thus, through the use of intelligent technologies, the proposed DCNN-based technique can aid
healthcare practitioners in evaluating the burn damage condition and providing appropriate treatments in the
shortest feasible time, remarkably reducing the unfavorable consequences of burns.

1. Introduction hypertrophic scarring, organ failure, and even death can ensue (Stoica,
Chircov, & Grumezescu, 2020).
Burns tend to be one of the most prevalent injuries in the world, The severity of a burn can be determined depending on the layers of
with consequences that can be deadly or cause a victim to suffer tissues damaged in the human body, with vascular, epidermal, dermal,
extremely if not treated appropriately. Catastrophic burn injuries are and muscles among the different tissues typically vulnerable to burn
extremely distressing and physically devastating traumas that impact wound (Laggner et al., 2022). The burn injury is usually classified into
approximately every major organs (Jeschke et al., 2011). According one of three categories by the healthcare professional: superficial (first
to a report of World Health Organization (WHO), burn injuries cause degree burns), superficial-partial or deep-partial burns (second degree
180,000 fatalities on average per year, while approximately 11 million burns), and full thickness burns (third degree burns), with each cate-
people were severely burnt and required medical treatment in the gory having different healing times and characteristics (Crouzet et al.,
year 2004 (Wang et al., 2018). Radiation, electricity, heat, excessive 2015). Burn wounds are dynamic and can develop as well as convert to
cold, chemical elements etc. can cause burn injuries, where treatments deeper wounds, making an accurate estimate of their depth and severity
must be ensured carefully according to its severity (Herndon, Zhang, & extremely challenging at an early stage (Rice & Orgill, 2021). Sufficient
Lineaweaver, 2022). With early and adequate treatment, the survival functional and structural investigations are necessary for precise burn
rates of burn victims can be considerably enhanced. Early burn wound intensity diagnosis. To measure the burn severity, modern techniques
excision, skin grafting, skin substitutes are typical treatment techniques such as laser Doppler imaging or medical evaluation under the supervi-
that can enhance the prognosis of severe burn patients by lowering fa- sion of experienced healthcare practitioners are necessary in traditional
tality rates and minimizing hospital stay days; whereas without correct clinical practice, but these procedures are constrained by factors such as
treatment at the right time poor wound healing, infection, discomfort, availability of the devices, distance, time, expense etc (Rangaraju et al.,

∗ Corresponding author.
E-mail addresses: [email protected] (S.A. Suha), [email protected] (T.F. Sanam).

https://doi.org/10.1016/j.mlwa.2022.100371
Received 18 April 2022; Received in revised form 8 June 2022; Accepted 16 June 2022
Available online 30 June 2022
2666-8270/© 2022 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/by-nc-nd/4.0/).
S.A. Suha and T.F. Sanam Machine Learning with Applications 9 (2022) 100371

Fig. 1. Burn depth classification.

2019). Because of these obstacles, the burn victims’ treatment process 2. Background study
may be delayed, resulting in a severe health deterioration. Furthermore,
standard manual methods for estimating burn severity, such as visual Burn damage is a typical occurrence in which a deep and exten-
inspection and physical assessment, not only cause delays, but have sive burn can result in catastrophic consequences such as sepsis from
also been proven to provide estimations that are only 50%–70% correct bacterial infection, shock from hypovolemia, massive fluid loss, organ
during the early days after a burn (Chauhan & Goyal, 2020). Early failure, and so on if not treated early (Shpichka et al., 2019). Burn
detection of burn depth severity becomes more challenging in remote injury can be classified based on its severity, depth of burn and size.
areas of least developed and developing countries, where healthcare Burns that just damage the top layer of the skin called epidermis
resources and facilities are scarce. are classified as superficial or first-degree burns in which the skin
turns red and the pain is short-lived; partial or intermediate thickness
Therefore, the purpose of this study is to propose an autonomous
known as second-degree burns are painful, drier, creates blisters, re-
classifier that can detect burn severity from real-time photographs of
quire dressing with wound care, and may scar, but they do not typically
the skin burn area and categorize it as first, second, or third degree
necessitate surgery; and finally a full-thickness or third-degree burns
burns. To attain this objective, a Deep Convolutional Neural Network
are dry that go through the entire dermis and is usually not painful
based machine learning classification model has been designed, trained,
due to nerve loss, but it does requires fluid resuscitation, protection
and tested using 1530 images of skin burns which intends to use
from infection, and unless the burn is extremely minor surgical care is
burn photos for detecting and categorizing patients’ burn severity. The essential (Jeschke et al., 2020; Noorbakhsh, Bonar, Polinski, & Amin,
significant contributions to achieve the goal of this study are listed 2021). Fig. 1 shows the illustration of categorization for burn depth
below. degrees according to its severity (Jeschke et al., 2020). Along with
clinical examination, Laser Doppler based techniques such as Laser
• A Deep Convolutional Neural Network (DCNN) based approach
speckle imaging (LSI) or Laser Speckle Contrast Analysis (LASCA);
has been proposed where a Convolutional Neural Network incor-
thermal imaging; Spatial Frequency Domain Imaging (SFDI) etc. are
porating different state-of-the-art techniques like transfer learning
among prominent techniques in medical field for correctly assessing
with pre-trained models and fine tuning on top of multiple convo-
perfusion in burns and burn depth detection (Ponticorvo et al., 2019).
lutional layers with hyperparameter tuning have been employed
Unfortunately, these procedures need the supervision of qualified spe-
for feature extraction from victims’ real-time burn photos; and
cialists, who may not be accessible at the time of the burn injury and
then a fully connected Artificial Neural Network has been used thus the burn wound progression may occur rapidly.
to classify them according to their severity into first, second and To solve this issue, several researches worldwide have applied
third degree burns. various computational techniques to automatically classify the burn
• The classification of the burn images according to their severity images and predict the severity of burn damage from the captured
have also been conducted through another approach employing injury images in real-time. In case of image classification based tasks,
traditional machine learning technique where the feature extrac- machine learning approaches are one of the most extensively utilized
tion have been conducted meticulously through several image and promising techniques, which generally analyze and retrieve critical
processing stages and then the classification has been conducted information from enormous quantities of heterogeneous data in order
using six types of conventional machine learning classifiers. to detect and classify anomalies autonomously (Tchito Tchapga, Mih,
• To validate the efficacy and potency of the proposed DCNN Tchagna Kouanou, Fozin Fonzin, Kuetche Fogang, Mezatio, & Tchiot-
technique, a comparative analysis have been conducted between sop, 2021). Therefore, employing various machine learning techniques
the traditional approach and suggested method. Also, three kinds for burn severity assessment is gaining traction nowadays. For exam-
of pre-trained models have been employed and tested in the CNN ple, the study referenced in Şevik, Karakullukçu, Berber, Akbaş, and
architecture with an aim to explore the best performing model in Türkyılmaz (2019), used 105 burnt photos to develop an automatic
this scenario. segmentation-based classification method to categorize burn images
into healthy skin, burned skin, and backgrounds for which they em-
The remaining paper is organized as follows: Section 2 explores the ployed four types of clustering approaches for image segmentation
background study, Section 3 demonstrates the research methodology; and then applied several traditional machine learning classification
Section 4 discusses the result analysis and findings; and Section 5 con- techniques with an aim to explore the best performing classifier. In the
cludes with a discussion and conclusion that highlights the important paper (Kuan, Chua, Safawi, Wang, & Tiong, 2017), an image mining
findings and future research plan. strategy was used to categorize different burn levels of captured burn

2
S.A. Suha and T.F. Sanam Machine Learning with Applications 9 (2022) 100371

images into three groups utilizing a comparative evaluation of 20 types 3. Methodology

of machine learning classification algorithms using both test dataset
and 10 fold cross validation approach. In the Study referenced (Yadav, In this research, a Deep Convolutional Neural Network (CNN) based
Sharma, Singh, & Goyal, 2019), the authors utilized 74 burn images to machine learning classification technique has been proposed, trained
develop a feature extraction model with several digital image process- and tested that aims to differentiate between the skin burn depth degree
ing steps and then classified the images into two classes using Support of patients utilizing the burn images. Here, the burnt photographs of the
Vector Machine classifier. Another related work had been explored in victims, captured using a digital camera or a mobile phone camera,
study Rowland et al. (2019), where a spatial frequency-domain imaging have been used as the study’s input, with the system predicting the
(SFDI) approach was combined with a support vector machine (SVM) degree of the skin burn. Along with the proposed CNN technique the
machine learning classifier to develop a model that can predict severity traditional image classification approach incorporating digital image
of progressive burns in a pig model and to estimate burn severity by processing and conventional machine learning classifiers have also
measuring the absorbance and scattering characteristics of burn tissue. been explored in this study for skin burn depth detection in order to
The work Hai, Triet, Thai, and Thuy (2017) had proposed a method assess a comparative performance and efficacy analysis of the suggested
for categorizing burn photos into the second, third, and fourth degrees technique. The framework of the research methodology followed in this
of severity, in which they used a combination of image processing study has been illustrated in Fig. 2. The stages involved are briefly
techniques concentrating on color feature extraction from the images discussed in the following sections.
and then SVM classifiers to categorize the images. Another work in Lee
et al. (2020) suggested a real-time technique for classification of burn 3.1. Data acquisition
depth employing moderate sample sizes based on ultrasound imaging,
in which the textural feature set is constructed using a grey-level co- The images utilized in this study are digital color images of skin
occurrence matrix (GLCM) derived from the ultrasound pictures of the burns captured with camera. There are total 1530 images of skin
burn tissue; and then utilizing a nonlinear support vector machine and burns from which 1440 images have been obtained from a publicly
kernel Fisher discriminant analysis, classification is accomplished in available repository of Kaggle (2022) and the rest of the images have
porcine skin tissue under four different burn scenarios. been collected from the burn units of three hospitals of Bangladesh
Recently a few studies also applied deep learning technique in this maintaining their patient privacy concerns. The photographs which
research area for classification and automatic severity detection of have been taken directly from the hospitals were captured using three
burns. For example, the study referenced in Karthik, Nath, and Veena distinct mobile phones containing digital camera. All of the pictures
from the open source repository, as well as those taken directly from
(2021) proposed a predictive model based on deep neural network,
the hospitals, are color pictures captured with an emphasis on the
recurrent neural network(RNN) and CNN to determine degree 1, degree
burn regions of the victims. Additionally, the images have optimal
2 and degree 3 of burn images depending on the severity of the burn
lighting conditions and good picture quality, allowing the burnt and
over a dataset of 104 images. In another study Pabitha and Vanathi
unburned parts to be distinguished from the background environment.
(2021), the authors presented a DenseMask Regional convolutional
According to their severity, the images are labeled as first degree,
neural network technique, which combined a Mask-region based con-
second degree, or third degree burns under the supervision of four
volution neural network with dense pose estimation for segmenting the
healthcare practitioner with two burn specialist doctors, where first
Region of Interest of a skin burn areas from images based on the sever-
degree burn photographs depicting less damage and third degree burn
ity of the burn damage. Another work proposed in the paper Abubakar,
being severely damaged. After categorizing them, it has been observed
Ugail, Smith, Bukar, and Elmahmudi (2020), applied deep neural net-
that the dataset contains 670 images of first degree, 347 images of
work with transfer learning using two pre-trained models ResNet50
second degree and 513 images of third degree burns.
and VGG16 for the feature extraction from images and then applied
Here, as the digital input images are all RGB colored images, the
SVM classification approach to classify the images into four categories
number of channels for each picture is 3. The pictures’ formats and sizes
which are healthy skin, first degree, second degree and third-degree vary since they were retrieved from various sources, rendering them
burns over 2080 RGB input images. The authors developed a deep inappropriate for using in predictive analytics. To resolve this issue all
learning-based system in work Liu, Yue, Cheng, Li, and Fu (2021), that the images have been resized to the shape (224 × 224 × 3) that contains
included precise burn area segmentation and burn depth labeling, as information about the row, column, and channel of the image and then
well as proposed a framework for enhanced burn area segmentation stored them in a 3D multichannel array. The labeling information for
and automated burn depth diagnosis based on deep learning methods. each image has also been stored in another one dimensional array
Also, the study referenced in Tran, Le, and Nguyen (2016) suggested which will be used as the target attribute for the machine learning
an approach for skin burn depth identification in which the pictures phase. The examples of one image from each category of input images
are pre-processed using Local Binary Pattern (LBP) operations based are shown in Fig. 3.
on recommendations of a burn specialist, and then an adaptive CNN
architecture is used to categorize burn images into four degrees based 3.2. Data augmentation
on their severity.
However, in the earlier studies rarely any researchers have focused Data augmentation, which comprises generating slightly modified
on the efficacy of employing CNN architecture with deep neural net- duplicates of images from the original training samples, is an effective
work over the traditional method of image classification with feature strategy for reducing data scarcity as well as improving performance
extraction via image processing techniques and conventional machine and minimizing forecast error (Shorten & Khoshgoftaar, 2019). In this
learning classifiers. Also, the utilization of transfer learning method study, three types of basic image manipulations have been applied
through various pre-trained models with fine tuning have been ex- to each images to formulate four additional training samples with
plored little. Therefore, this study proposes a CNN architecture that image augmentation process which includes geometric transformations
integrates several state-of-art techniques such as transfer learning with (random flipping, rotating and shifting); contrast enhancement and
fine tuning to effectively classify the burn images according to their sharpening with noise removal. As a consequence, the dataset without
severity; and also it conducts the classification analysis through tra- augmentation has 1530 images, and a new augmented dataset con-
ditional approaches to have a comparative study with the proposed taining 6120 photographs of burnt areas has been generated applying
method. augmentation techniques.

3
S.A. Suha and T.F. Sanam Machine Learning with Applications 9 (2022) 100371

Fig. 2. Framework of research methodology.

a more evenly dispersed histogram, resulting in a sharper and

clearer image for viewers. As histogram equalization operation
cannot be done in colored RGB image space, thus the images are
converted to Hue Saturation Value (HSV) format, apply histogram
equalization in that HSV image and then again converted back to
RGB format which will resulted into an enhanced image. The next
stage is ‘‘Noise Removal’’ for which as an image restoration phase
of noise reduction, a median filter approach is used; which is a
Fig. 3. Example of first, second and third degree burnt input images.
commonly used nonlinear filter with superior edge maintaining
capabilities and the potential to minimize impulsive noise (Zhu &
Huang, 2012). Following that, the required amount of ‘‘Morpho-
3.3. Applying machine learning techniques logical Erosion’’ and ‘‘Morphological Dilation’’ operations have
been performed where erosion reduces and dilation enhances the
In this research, two types of machine learning techniques have pixel numbers on the edges of objects in the images. The final
been employed to classify the burnt images of patients, where the image processing step applied here is ‘‘Image Segmentation’’ to
first one is with several conventional machine learning classifiers and divide the images into different significant areas. For this, here
the second one is with the proposed CNN architecture incorporating k-means unsupervised clustering algorithm has been applied that
several state-of-art techniques like transfer learning, fine tuning etc. segments the interest area of burn from the background and thus
The Google Colaboratory has been utilized here as the platform for separates the affected area from the healthy skin (Kumari, Prasad,
execution, with the scikit-learn and TensorFlow packages from Python. & Mounika, 2019).
The two approaches are described hereafter. • Split into Train and Test Data: After the specified image pro-
cessing stages are completed, the set of processed pictures are
3.3.1. Traditional approach turned into a single dimension array by using OpenCV python
Training with a large number of high-resolution images necessitates function ‘FLATTEN’. Each column in this array represents an at-
the processing of large volumes of data, which is troublesome for tribute that contains the image’s pixel values. This array with the
most traditional machine learning models (Lézoray, Charrier, Cardot, & pixel values has been used as the training’s input features, with
Lefèvre, 2008). For this reason, it is required to perform the appropriate the one-dimensional array containing the labeling information for
digital image processing steps to extract and highlight the key informa- each image functioning as the target attribute. Finally, to use this
tion from the images before utilizing them in the conventional machine processed dataset in the machine learning classification models,
learning classifiers. Thus, in this approach the images are initially pre- the dataset is divided into 70% train data and 30% test data.
processed with digital image processing steps; then splitted into train • Training with Traditional Machine Learning Classifiers: This
and test data; and finally employed in the traditional machine learning study’s predictive analysis is a multiclass classification problem
classifiers. where the images can be classified into three classes. Therefore,
six different types of conventional classifiers have been employed
• Digital Image Processing Steps: After the image acquisition here with an aim to classify the images into first, second or
and resizing, the first step of digital image processing that has third degree burn. The classification models are: Logistic Re-
been applied here is the ‘‘Image Enhancement’’. This is achieved gression classifier, K-Nearest Neighbor(KNN) classifier, Support
here via the histogram equalization approach, which enhances Vector Machine (SVM) classifier, Decision Tree classifier, Random
the visual perception of information in photographs by giving Forest classifier and Multi-layer Perceptron classifier.

4
S.A. Suha and T.F. Sanam Machine Learning with Applications 9 (2022) 100371

VGG16: The first pretrained model that has been used in this
study is VGG16 containing 16 convolution layers with different
weights which follows the architecture of containing 3x3 filters
with a stride 1 and the same padding and maxpool layer of a
2x2 filter with a stride 2 as well as contains two fully connected
dense layers in the end, followed by a softmax for output. This
network is quite vast, with approximately 138 million parameters.
This model has provided best accuracy of 90.1% with Imagenet
dataset (Tammina, 2019)
MobileNet: MobileNet is a basic yet efficient convolutional neu-
ral network that is commonly used for mobile vision applications
Fig. 4. Basic framework of transfer learning method. has been employed in this study as one of the pretrained models.
The MobileNet Architecture is built on the foundation of two
types of convolution layers: depthwise separable convolution and
Therefore, in the traditional machine learning approach these above pointwise convolution layers. The depthwise layer is the map
mentioned steps are followed in order to classify the images according of a single convolution upon every input channel individually,
to their burn depth severity. followed by pointwise convolution layers with a 1x1 filter mul-
tiplication operation to merge the depthwise layer’s feature map
3.3.2. Proposed CNN approach results. This model has provided best accuracy of 89.5% with
Convolutional Neural Networks (CNN) are indeed a type of deep Imagenet dataset (Harjoseputro, Yuda, Danukusumo, et al., 2020)
neural network with multiple consecutive layers that have shown to ResNet50: The third type of pretrained model that has been
perform efficiently in a range of image processing, classification, and employed here is ResNet50 which had been introduced based
segmentation tasks (Kim, Jung, Park, Lee, & Ahn, 2022). The ad- on the concept of residual network to solve the problem of
vantage of employing CNN is that, it can successfully tackle picture vanishing gradient in deep neural network. ResNet50 comprises
classification problems with higher accuracy since it matches the data 50 layers of residual networks, which contains distinct groups
point distribution in the picture throughout the neural network training of identical layers, with identity blocks connecting two layers of
process and can directly utilize the feature maps from the convolutional varied sizes in between. The Skip Connections between layers
layers. As a consequence, substantial features from the images can be function adds prior layer outputs to the outcomes of stacked
extracted autonomously without the need for explicit image processing layers, allowing for far deeper network training than formerly
operations (Dabeer, Khan, & Islam, 2019). feasible. This model has provided best accuracy of 94.5% with
In this research, a CNN architecture with different layers was rig- Imagenet dataset (Mukti & Biswas, 2019)
orously built with the goal of classifying burn images. After data • Convolution Layer with padding and striding CNN’s essential
acquisition from the repository, the dataset is divided into 70% train and fundamental component layer is the next layer which is
and 30% test data in this approach; and then the CNN model is built called ‘‘Convolution layer’’ that consists of a series of kernels or
to use this dataset for training. ‘‘Sequential’’ is the model type that has filters that are mostly smaller in size than the original training
been employed here to construct the CNN architecture layer by layer
picture and whose parameters must be trained throughout time of
with the help of ‘‘add()’’ function for adding each layer. The layers of
learning and convolve with the real image (Mostafa & Wu, 2021).
the CNN architecture are discussed hereafter.
Convolutional layers use a collection of convolutional kernels to
• Transfer Learning with Pretrained models The architecture’s extract features, which perform convolution operations on the
first layer after data augmentation is based on transfer learning input dataset or feature maps through intermediate layers (Ren
with fine tuning, a powerful machine learning strategy that aims et al., 2022).
to improve target learners’ performance on intended domains by In this study after the transfer learning layer with pre-trained
transferring information from the relevant pre-trained models and model, the 2 dimensional convolution layer (Conv2D) operation
then making small adjustment to get the prediction (Zhuang et al., has been performed two times where the first layer contained 128
2020). Here, the pre-trained model’s fully connected or dense nodes and second one 64 nodes. In both cases, the kernel or filter
layer from the source task has been removed using ‘‘𝑖𝑛𝑐𝑙𝑢𝑑𝑒 𝑡𝑜𝑝 = size for convolution has been considered to be 3 × 3. Moreover,
𝐹 𝑎𝑙𝑠𝑒’’ operation and retrained them for the target task using ‘‘padding’’ and ‘‘striding’’ techniques have been used here in the
some more convolution and dense layers with hyper-parameter convolution layer to enhance the accuracy of the output; where
tuning. The basic framework followed for transfer learning has padding adds one additional layer to the outer image and striding
been illustrated in 4. manages the space between two consecutive kernel locations.
In this study, three different types of pre-trained models have The value of padding in this CNN architecture is ‘‘same’’ which
been trained, and tested in the transfer learning layer to evaluate indicates padding the input image with zeros evenly to the outer
which model provides better performance for classification. The side. The value of stride is 1, which signifies the output size
models are: VGGNet16 model; MobileNet model; and ResNet50 after convolution operation with the filter will be the same as
model (Keras, 2022). The Tensorflow and Keras library contains the input size. Then, an activation function is employed in this
pre-trained models, where the weights are derived from three-
deep neural network to feed a weighted sum of input signals
channel pictures. These pre-trained models have been trained us-
through and with the result being utilized as an input to the next
ing millions of images from the ImageNet dataset to predict over
layer. In this CNN model, ‘‘Softmax’’ activation function has been
1000 classes; consequently, utilizing pre-trained models allows a
employed which tends to produce useful outcomes that can be ap-
new model to converge quicker and perform better on a smaller
plied to multiclass classification problems and provides an output
dataset by leveraging features learnt on a bigger dataset (Arias-
between 0 to 1 range representing the probability classification
Garzón et al., 2021). The pretrained models are here used without
outcome (Sharma, 2019). The mathematical representation of the
their final classification layer, which will aid in the transforma-
softmax activation function has been shown in Eq. (1).
tion of images from a new domain job based on its hidden states,
allowing for the extraction of features from a new domain work 𝑒𝑥𝑝(𝑎𝑖 )
𝑆𝑜𝑓 𝑡𝑚𝑎𝑥 𝐴𝑐𝑡𝑖𝑣𝑎𝑡𝑖𝑜𝑛, 𝑆(𝑎𝑖 ) = ∑𝑛 (1)
while leveraging information from a source-domain task. 𝑗=1 𝑒𝑥𝑝(𝑎𝑗 )

5
S.A. Suha and T.F. Sanam Machine Learning with Applications 9 (2022) 100371

• Pooling Layer: The ‘‘Pooling Layer’’ is the following layer in (recall) and F1 score. The performance indicators are mostly based on
this study’s CNN design, which declines the spatial size of the a comparison of forecasted and real values from the training dataset,
picture while retaining crucial data to progressively reduce the which is separated into four groups: True Positive (TP) that refers to
computations and spatial size in the complex and large neural a situation in which both the true and predicted values are positive;
network (Yamashita, Nishio, Do, & Togashi, 2018). Here, a ‘‘Max True Negative (TN) in which the original value is negative and also the
Pooling’’ technique has been used to obtain the maximum value anticipated value is negative.; False Positive (FP) a scenario in which
for each patch of the feature map. The pooling layer has been the real value is negative but the anticipated result from the training
employed two times after each of the convolution layers with the is positive and finally the actual value is positive, but the forecasted
first max-pool layer having a pooling window size of result is negative, leading in a False Negative (FN). The performance
metrics can be expressed as following Eqs. (2),(3),(4) and (5) based on
4×4 these assessments:
and the second max-pool layer having a pooling window size of 𝑇𝑃 + 𝑇𝑁
𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 = (2)
𝑇𝑃 + 𝑇𝑁 + 𝐹𝑃 + 𝐹𝑁
2 × 2.
𝑇𝑃
𝑃 𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 = (3)
• Dropout Layer: The ‘‘Dropout Layer’’ is the next layer, which is 𝑇𝑃 + 𝐹𝑃
a regularization method that significantly reduces overfitting and 𝑇𝑃
speed up the learning process by changing input data to 0 for 𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦(𝑅𝑒𝑐𝑎𝑙𝑙) = (4)
𝑇𝑃 + 𝐹𝑁
ignoring some nodes at a predetermined frequency rate at each
2 ∗ (𝑃 𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 ∗ 𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦)
step during the training process (Nandini, Kumar, & Chidananda, 𝐹 1 − 𝑠𝑐𝑜𝑟𝑒 = (5)
𝑃 𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 + 𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦
2021). To conduct the regularization, the dropout rate in this CNN
model has been set to 0.2 in the dropout layer. 4. Result analysis
• Flatten Layer: The ‘‘Flatten layer’’ is the next layer, which con-
verts the multi-dimensional outputs of the preceding layer into a 4.1. Findings from the traditional machine learning approach
one dimensional array to be used in the next classification layer.
The input layer of the classifying neural network is built using this Employing the traditional ML approach, the images have been pre-
one-dimensional array, with the components of the array being processed with required digital image processing stages before applying
provided to each neuron. Therefore, this layer works as the bridge them into ML classification algorithms. One of the examples of a
between the convolution and dense layer. The most significant burnt image with each steps of image processing technique has been
features from the images are also extracted in this layer as a result illustrated in Fig. 5. From the figure it is visible that, after applying
of the previous layers. the digital image processing steps the affected burnt area is more
• Fully Connected Layer: The ‘‘Fully Connected Layer’’ also known prominently detected in the final segmented image in comparison to
as ‘‘Dense Layer’’ is the classifier and final layer of this CNN the original image. Similarly, all the images are processed in this way
architecture. This layer is at the bottom of the CNN model, and then employed to the machine learning classifier as train and test
and every neuron inside it is connected to every neurons in the image data.
previous and forward layers, adopting the standard multiple-layer Both types of datasets have been used to train, test, and assess
perceptron and feed forward neural network technique (Alzubaidi the six types of standard machine learning classifiers: dataset without
et al., 2021). As the classification problem here has three classes, image augmentation having 1530 images and dataset with image aug-
the last layer of this dense network will contain three nodes to mentation containing 6120 images. The performances of the six types
provide the classification prediction, one for each possible out- of machine learning classifiers with different performance metrices
come. Here also ‘‘softmax’’ activation function has been utilized to employing two types of dataset have been shown in Table 1. From
generate the result from the output neurons possessing the highest the performance analysis in Table 1, it can be observed that, for all
possibility. the classifiers the performances enhance when they are trained with
• Compiling Model: Finally, the CNN model has been compiled augmented dataset. For example, the accuracy of Decision Tree classi-
using three parameters: optimizer, metrics and loss. The optimizer fier without augmented dataset is 70.6%, which becomes 71.4% after
utilized here is the stochastic gradient descent (SGD) optimizer, employing dataset with augmented images. This results indicate that,
which is used to regulate the learning rate. Here the learning the prediction performances of the machine learning models improves
rate is 0.01. ‘‘Accuracy’’ has been used as the metrics of the when an increased amount of data are utilized to train the models.
model for assessing the performance of the training. And lastly, Furthermore, among all the classifiers, Random Forest classifica-
for evaluating the loss ‘‘𝑐𝑎𝑡𝑒𝑔𝑜𝑟𝑖𝑐𝑎𝑙_𝑐𝑟𝑜𝑠𝑠𝑒𝑛𝑡𝑟𝑜𝑝𝑦’’ function is used tion model comparatively outperforms others in terms of performing
as the problem here is a multi-class classification problem. The with highest accuracy being 77.3% without augmented dataset and
lower value of loss indicates better performance. 80.4% with augmented images whereas the Multi-Layer Perceptron
provides the least accuracy 53.0% without augmented data and 56.0%
To explore the best performing classification model, the CNN archi- employing augmented image dataset.
tecture formulated in the above mentioned way is trained with the
dataset in four different methods and hyperparameter optimization : (i) 4.2. Findings from the proposed CNN approach
without employing transfer learning layer; (ii) incorporating ‘‘VGG16’’;
(iii) incorporating ‘‘MobileNet’’; and (iv) incorporating ‘‘ResNet50’’ pre- The proposed CNN architecture has been slightly modified with
trained model for transfer learning layer. For each type the training has hyperparameter tuning in four different ways to explore the best per-
been conducted in 30 epochs and then the models have been evaluated forming model for burn depth prediction where each method has
with test dataset. executed over 30 epochs for training. Fig. 6 shows the accuracy and
loss per epoch of the training with 30 epochs where Fig. 6(a) shows
3.4. Performance analysis the accuracy and loss for the CNN model without transfer learning,
6(b) shows CNN model with VGG16, 6(c) shows CNN model with
To evaluate the efficacy of the predictive analysis, the performances MobileNet and 6(d) shows CNN model with ResNet50. From the figure
of both types of machine learning approaches are assessed using four it is apparent that for all the models the accuracy gradually increases
performance metrics which are the Accuracy, Precision, Sensitivity and the loss decreases. Also, the accuracies and losses fluctuate slightly

6
S.A. Suha and T.F. Sanam Machine Learning with Applications 9 (2022) 100371

Fig. 5. Findings from Image Processing Stages.

Table 1
Performance analysis results obtained from the test data employing traditional machine learning approach with and without
augmented dataset.
Machine Learning Models Without Augmented Dataset With Augmented Dataset
Acc. Prec. Rec. F1-sc. Acc. Prec. Rec. F1-sc.
Random Forest Classifier 0.773 0.784 0.734 0.744 0.804 0.81 0..807 0.809
Support Vector Machine 0.767 0.762 0.740 0.747 0.78 0.785 0.781 0.778
Decision Tree 0.706 0.692 0.692 0.692 0.714 0.715 0.725 0.712
K-Nearest Neighbor 0.661 0.796 0.571 0.517 0.696 0.698 0.698 0.684
Logistic Regression 0.642 0.619 0.601 0.601 0.653 0.654 0.655 0.623
Multi-Layer Perceptron 0.530 0.595 0.580 0.524 0.56 0.565 0.564 0.543

Table 2 the other models with 95.63% accuracy. Moreover, it is also noticeable
Performance analysis results obtained from the test data with proposed CNN machine that, the performance of the classification model enhances significantly
learning approach.
while employing transfer learning technique as the CNN model without
CNN Architecture Accuracy Precision Recall F1-score
transfer learning has acquired only 72.01% accuracy.
CNN without Transfer Learning 0.7715 0.762 0.749 0.780 The comparative accuracy analysis of employing traditional and
CNN with Transfer Learning- (VGG16) 0.9580 0.96 0.95 0.95
CNN with Transfer Learning- (MobileNet) 0.8992 0.94 0.89 0.90
proposed CNN technique with different machine learning models have
CNN with Transfer Learning- (ResNet50) 0.9325 0.93 0.91 0.92 been illustrated in Fig. 7. From the comparative performance analysis it
is evident that, the accuracy of the models with traditional approaches
is much lower in comparison to the CNN techniques. Furthermore, the
traditional technique entails several tedious image processing phases
after 25 epochs for all the models, indicating that the models have that need meticulous adjustments whereas the CNN technique can
converged. Here, among all the models Fig. 6(b) illustrating accuracy provide excellent performance without any explicit image processing
and loss of CNN with transfer learning employing VGG16 pretrained steps as the neural network itself extract the significant features from
model shows the best performance where the accuracy and loss con- the images at the time of training. Therefore, the suggested method,
verge comparatively earlier being stable at almost 99% accuracy and which integrates transfer learning with pre-trained VGG16 model, can
0% loss. On the other hand, Fig. 6(a) representing the accuracy and loss be used effectively in clinical practice to estimate the depth of burn
of CNN model without transfer learning shows the worst performance from captured photos of skin burn injury.
with frequent fluctuations of accuracy and losses as well as the accuracy
being 4.3. Limitations and future perspectives
Table 2 shows the performances of the proposed CNN technique on
the test dataset with four methods; which consists of without including One of the research’s limitations is that, although the study has
transfer learning from pretrained models as well as including three employed photographs from a variety of archives, almost all of them
different pre-trained models as transfer learning process. From the have precise views of the victims’ burn regions. It is recommended that
table, it is apparent that all of the CNN approaches with the proposed clear and sharp photographs should be captured focusing on the burn
technique have achieved substantially higher accuracy, with the trans- region of the patient that is distinguishable from the background, with
fer learning model using the ‘‘VGG16’’ pre-trained model outperforming decent lighting, reduced shadows, and moderate or good color picture

7
S.A. Suha and T.F. Sanam Machine Learning with Applications 9 (2022) 100371

Fig. 6. Accuracy and loss obtained per epoch from (a) CNN without transfer learning (b) CNN with VGG16 pretrained model for transfer learning (c) CNN with MobileNet
pretrained model for transfer learning (d) CNN with ResNet50 pretrained model for transfer learning.

8
S.A. Suha and T.F. Sanam Machine Learning with Applications 9 (2022) 100371

healthcare facilities where there is limited resources to conduct appro-

priate clinical diagnosis for detecting skin burn severity. However, this
study may be expanded in the future by integrating additional samples,
which will allow for the distinction of superficial-partial and deep-
partial burns, as well as the estimation of the Total Body Surface Area
(TBSA%) of burn for improved clinical assessment for burn patients.

CRediT authorship contribution statement

Sayma Alam Suha: The idea of this article was developed, Back-
ground study was conducted, The methodological framework was pro-
posed, The simulation and result analysis was carried out, Part of
M.Sc. thesis work, Interpreting the data. Tahsina Farah Sanam: The
idea of this article was developed, The methodological framework was
proposed, Toward rewriting the entire draft article to prepare it for
publication, Supervision, Interpreting the data.

Fig. 7. Comparative accuracy analysis. Declaration of competing interest

The authors declare that they have no known competing finan-

quality for this study. However, in a real-time setting, there may be cial interests or personal relationships that could have appeared to
photographs acquired by users with blurry views of burns and lower influence the work reported in this paper.
picture quality that may become difficult to classify in terms of burn
severity level. Thus, in future, the research team intends to perform Data availability
some practical post predictive analysis with different sample of data,
focusing on this flaw in the deployment of a real-time predictive model The dataset generated and analyzed during the current study are
in clinical sector based on the proposed CNN classifier to ensure that available from the corresponding author on reasonable request.
all instances can be reliably anticipated to predict the burn severity of
patients. Acknowledgments
Moreover, this study has utilized only three types of pretrained
models for transfer learning in the CNN architecture. Also, for intel- The authors in this research are grateful to Dr.Nabil Sharik (MBBS,
ligent clinical applications, an explainable AI plays a significant role FCPS, Assistant Surgeon, Sadar, Gopalganj) Dr. Atikur Rahman (MBBS,
providing an explanation with proper justification of predictions made MO in NiCU & PiCU, M R Khan Shishu Hospital, Bangladesh) and Dr.
by the AI systems, which has not been incorporated in this study. Fariba Fareen (MBBS, MO, Mymensingh Medical College, Bangladesh)
Therefore, in future, the researchers plan to employ more varieties of for enriching the study with clinical guidelines.
potential pretrained models with enhanced convolution layers as well
as aggregate the techniques of XAI (explainable AI) with the proposed
References
methodology in order to explore a better performing classifier with
clinical validation. Abubakar, A., Ugail, H., Smith, K. M., Bukar, A. M., & Elmahmudi, A. (2020). Burns
depth assessment using deep learning features. Journal of Medical and Biological
5. Discussion and conclusion Engineering, 40(6), 923–933.
Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O.,
et al. (2021). Review of deep learning: Concepts, CNN architectures, challenges,
This research has proposed a DCNN based model with transfer
applications, future directions. Journal of Big Data, 8(1), 1–74.
learning and fine tuning for effective identification of skin burn degrees Arias-Garzón, D., Alzate-Grisales, J. A., Orozco-Arias, S., Arteaga-Arteaga, H. B., Bravo-
according to their severity from real-time burn images of victims. The Ortiz, M. A., Mora-Rubio, A., et al. (2021). COVID-19 detection in X-ray images
architecture employs transfer learning from pretrained models and then using convolutional neural networks. Machine Learning with Applications, 6, Article
multiple convolutional layers with hyperparameter tuning for feature 100138.
Chauhan, J., & Goyal, P. (2020). BPBSAM: Body part-specific burn severity assessment
extraction from the images which are then utilized in a fully connected
model. Burns, 46(6), 1407–1423.
feed forward neural network to classify the images into three cate- Crouzet, C., Nguyen, J. Q., Ponticorvo, A., Bernal, N. P., Durkin, A. J., & Choi, B.
gories. To validate and evaluate the efficacy of the proposed technique, (2015). Acute discrimination between superficial-partial and deep-partial thickness
the traditional approach incorporating digital image processing and burns in a preclinical model with laser speckle imaging. Burns, 41(5), 1058–1063.
conventional machine learning classifiers has also been applied to solve Dabeer, S., Khan, M. M., & Islam, S. (2019). Cancer diagnosis in histopathological
image: CNN based approach. Informatics in Medicine Unlocked, 16, Article 100231.
this multi-class classification problem. Accuracy, precision, recall and
Hai, T. S., Triet, L., Thai, L., & Thuy, N. (2017). Real time burning image classification
F1-scores are the metrices that have been utilized to conduct the perfor- using support vector machine. EAI Endorsed Transactions on Context-Aware Systems
mance analysis. The findings from comparative analysis indicates that and Applications, 4(12).
the proposed technique shows much higher accuracy in comparison Harjoseputro, Y., Yuda, I., Danukusumo, K. P., et al. (2020). MobileNets: Efficient
to the traditional approach and also incorporating transfer learning convolutional neural network for identification of protected birds. IJASEIT (Interna-
tional Journal on Advanced Science, Engineering and Information Technology), 10(6),
outperforms other techniques in terms of accuracy.
2290–2296.
Therefore, the suggested DCNN based computational model can Herndon, D., Zhang, F., & Lineaweaver, W. (2022). Metabolic responses to severe burn
assist medical practitioners and healthcare providers in evaluating the injury. Annals of Plastic Surgery, 88(2), S128–S131.
injury condition and suggesting suitable therapy in a quickest possible Jeschke, M. G., van Baar, M. E., Choudhry, M. A., Chung, K. K., Gibran, N. S., &
time depending on the degree of the skin burn. Through the utilization Logsetty, S. (2020). Burn injury. Nature Reviews Disease Primers, 6(1), 1–25.
Jeschke, M. G., Gauglitz, G. G., Kulp, G. A., Finnerty, C. C., Williams, F. N., Kraft, R., et
Artificial intelligence and advanced technologies, this approach can
al. (2011). Long-term persistance of the pathophysiologic response to severe burn
also provide telemedicine support to diagnose and treat patients re- injury. PLoS One, 6(7), Article e21245.
motely, especially in rural areas of least or developing countries where Kaggle (2022). Skin burn dataset. URL https://www.kaggle.com/shubhambaid/skin-
professionals may be scarce. This method can also be utilized in the burn-dataset.

9
S.A. Suha and T.F. Sanam Machine Learning with Applications 9 (2022) 100371

Karthik, J., Nath, G. S., & Veena, A. (2021). Deep learning-based approach for skin Ren, Z., Qian, K., Dong, F., Dai, Z., Nejdl, W., Yamamoto, Y., et al. (2022). Deep
burn detection with multi-level classification. In Advances in computing and network attention-based neural networks for explainable heart sound classification. Machine
communications (pp. 31–40). Springer. Learning with Applications, Article 100322.
Keras, T. (2022). Keras documentation: Keras applications-pretrained models. URL Rice, P. L., & Orgill, D. (2021). Assessment and classification of burn injury. UpToDate,
https://keras.io/api/applications/. [Internet].
Kim, H., Jung, W.-K., Park, Y.-C., Lee, J.-W., & Ahn, S.-H. (2022). Broken stitch de- Rowland, R. A., Ponticorvo, A., Baldado, M. L., Kennedy, G. T., Burmeister, D.
tection method for sewing operation using CNN feature map and image-processing M., Christy, R. J., et al. (2019). Burn wound classification model using spatial
techniques. Expert Systems with Applications, 188, Article 116014.
frequency-domain imaging and machine learning. Journal of Biomedical Optics,
Kuan, P., Chua, S., Safawi, E., Wang, H., & Tiong, W. (2017). A comparative study
24(5), Article 056007.
of the classification of skin burn depth in human. Journal of Telecommunication,
Şevik, U., Karakullukçu, E., Berber, T., Akbaş, Y., & Türkyılmaz, S. (2019). Automatic
Electronic and Computer Engineering (JTEC), 9(2–10), 15–23.
classification of skin burn colour images using texture-based feature extraction. IET
Kumari, C. U., Prasad, S. J., & Mounika, G. (2019). Leaf disease detection: feature
extraction with K-means clustering and classification with ANN. In 2019 3rd inter- Image Processing, 13(11), 2018–2028.
national conference on computing methodologies and communication (pp. 1095–1098). Sharma, O. (2019). A new activation function for deep neural network. In 2019
IEEE. International Conference on Machine Learning, Big Data, Cloud and Parallel Computing
Laggner, M., Lingitz, M.-T., Copic, D., Direder, M., Klas, K., Bormann, D., et al. (2022). (pp. 84–86). IEEE.
Severity of thermal burn injury is associated with systemic neutrophil activation. Shorten, C., & Khoshgoftaar, T. M. (2019). A survey on image data augmentation for
Scientific Reports, 12(1), 1–10. deep learning. Journal of Big Data, 6(1), 1–48.
Lee, S., Ye, H., Chittajallu, D., Kruger, U., Boyko, T., Lukan, J. K., et al. (2020). Shpichka, A., Butnaru, D., Bezrukov, E. A., Sukhanov, R. B., Atala, A., Burdukovskii, V.,
Real-time burn classification using ultrasound imaging. Scientific Reports, 10(1), et al. (2019). Skin tissue regeneration for burn injury. Stem Cell Research & Therapy,
1–13. 10(1), 1–16.
Lézoray, O., Charrier, C., Cardot, H., & Lefèvre, S. (2008). Machine learning in image Stoica, A. E., Chircov, C., & Grumezescu, A. M. (2020). Hydrogel dressings for the
processing. EURASIP Journal on Advances in Signal Processing, 2008, 1–2. treatment of burn wounds: an up-to-date overview. Materials, 13(12), 2853.
Liu, H., Yue, K., Cheng, S., Li, W., & Fu, Z. (2021). A framework for automatic burn Tammina, S. (2019). Transfer learning using VGG-16 with deep convolutional neural
image segmentation and burn depth diagnosis using deep learning. Computational network for classifying images. International Journal of Scientific and Research
and Mathematical Methods in Medicine, 2021.
Publications (IJSRP), 9(10), 143–150.
Mostafa, S., & Wu, F.-X. (2021). Diagnosis of autism spectrum disorder with convolu-
Tchito Tchapga, C., Mih, T. A., Tchagna Kouanou, A., Fozin Fonzin, T., Kuetche Fo-
tional autoencoder and structural MRI images. In Neural engineering techniques for
gang, P., Mezatio, B. A., et al. (2021). Biomedical image classification in a big data
autism spectrum disorder (pp. 23–38). Elsevier.
architecture using machine learning algorithms. Journal of Healthcare Engineering,
Mukti, I. Z., & Biswas, D. (2019). Transfer learning based plant diseases detection
2021.
using ResNet50. In 2019 4th international conference on electrical information and
communication technology (pp. 1–6). IEEE. Tran, H. S., Le, T. H., & Nguyen, T. T. (2016). The degree of skin burns images
Nandini, G. S., Kumar, A. S., & Chidananda, K. (2021). Dropout technique for image recognition using convolutional neural network. Indian Journal of Science, 9(45),
classification based on extreme learning machine. Global Transitions Proceedings, 1–6.
2(1), 111–116. Wang, Y., Beekman, J., Hew, J., Jackson, S., Issler-Fisher, A. C., Parungao, R., et al.
Noorbakhsh, S. I., Bonar, E. M., Polinski, R., & Amin, M. S. (2021). Educational case: (2018). Burn injury: Challenges and advances in burn wound healing, infection,
Burn injury—Pathophysiology, classification, and treatment. Academic Pathology, 8, pain and scarring. Advanced Drug Delivery Reviews, 123, 3–17.
Article 23742895211057239. Yadav, D., Sharma, A., Singh, M., & Goyal, A. (2019). Feature extraction based machine
Pabitha, C., & Vanathi, B. (2021). Densemask RCNN: A hybrid model for skin burn learning for human burn diagnosis from burn images. IEEE Journal of Translational
image classification and severity grading. Neural Processing Letters, 53(1), 319–337. Engineering in Health and Medicine, 7, 1–7.
Ponticorvo, A., Rowland, R., Baldado, M., Burmeister, D. M., Christy, R. J., Bernal, N. Yamashita, R., Nishio, M., Do, R. K. G., & Togashi, K. (2018). Convolutional neural
P., et al. (2019). Evaluating clinical observation versus spatial frequency domain networks: An overview and application in radiology. Insights Into Imaging, 9(4),
imaging (SFDI), laser speckle imaging (LSI) and thermal imaging for the assessment 611–629.
of burn depth. Burns, 45(2), 450–460.
Zhu, Y., & Huang, C. (2012). An improved median filtering algorithm for image noise
Rangaraju, L. P., Kunapuli, G., Every, D., Ayala, O. D., Ganapathy, P., & Mahadevan-
reduction. Physics Procedia, 25, 609–616.
Jansen, A. (2019). Classification of burn injury using Raman spectroscopy and
Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., et al. (2020). A comprehensive
optical coherence tomography: An ex-vivo study on porcine skin. Burns, 45(3),
survey on transfer learning. Proceedings of the IEEE, 109(1), 43–76.
659–670.

10