See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/355361624

An MRI-based deep learning approach for efficient classification of brain

tumors

Article in Journal of Ambient Intelligence and Humanized Computing · October 2021

DOI: 10.1007/s12652-021-03535-9

CITATIONS READS

53 958

5 authors, including:

Huang Jianjun Hafeez UL Haq

Shenzhen University Helmholtz Centre for Environmental Research
109 PUBLICATIONS 1,351 CITATIONS 12 PUBLICATIONS 165 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Huang Jianjun on 15 December 2023.

The user has requested enhancement of the downloaded file.

Journal of Ambient Intelligence and Humanized Computing
https://doi.org/10.1007/s12652-021-03535-9

ORIGINAL RESEARCH

An MRI‑based deep learning approach for efficient classification

of brain tumors
Ejaz Ul Haq1 · Huang Jianjun1 · Kang Li1 · Hafeez Ul Haq2 · Tijiang Zhang3

Received: 29 November 2020 / Accepted: 6 October 2021

© The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2021

Abstract
Efficient and reliable identification and classification of brain tumors from imaging data is essential in the diagnosis and
treatment of brain cancer cells. Magnetic resonance imaging (MRI) is the most commonly used imaging modality in the
analysis of infected brain tissue. However, manual segmentation requires significant time to process data produced by mag-
netic resonance imaging. In this study, we present two fast and proficient brain tumor identification techniques based on deep
convolutional neural networks (CNNs) using magnetic resonance imaging data for the effective detection and classification of
different types of brain tumors. We use two publicly available datasets from Figshare and BraTS 2018, and apply conditional
random fields to eliminate forged outputs, considering spatial information on fine segmentation tasks. The first proposed
architecture, based on the Figshare dataset, classifies brain tumors as gliomas, meningiomas, or pituitary tumors. The second
architecture differentiates between high- and low-grade gliomas (HGG and LGG, respectively). An intensity normalization
method is also investigated as a pre-processing step, which proves highly effective at detection and classification of brain
tumors in combination with data augmentation techniques. The Figshare and BraTS 2018 datasets included 3062 and 251
images, respectively. The experimental results demonstrate an accuracy of 97.3% and a dice similarity coefficient (DSC)
95.8% on the task of classifying brain tumor as gliomas, meningiomas, or pituitary tumors achieved by the first proposed
CNN architecture, while second proposed CNN architecture achieved an accuracy of 96.5% with a DSC of 94.3% on the
task of classifying glioma grades as HGG or LGG. Experimental results reveal that our proposed model attained improved
performance and increased classification accuracy compared to state-of-the-art methods.

Keywords Segmentation process · Brain tumor · Deep convolutional neural network · Data augmentation · Magnetic
resonance images

1 Introduction segments, and have been widely applied in medical image

analysis. In the recent era, brain cancer has been identified
Image processing is among the most critical subjects in com- as among the most frequently mortal brain diseases. Gliomas
puter vision, and has been widely utilized in the medical are considered the most commonly occurring brain tumors,
field for the effective identification and treatment of diseases, and have the highest fatality rate (Bauer et al. 2013). Arising
as well as a general approach to solving real-time problems primarily in the brain or spinal cord, gliomas are categorized
in all aspects of engineering science. Image segmentation into two sub-grades, including low-grade gliomas (LGGs)
methods are commonly used to divide images into multiple and high-grade gliomas (HGGs). High-grade gliomas are
considered more destructive and penetrative, and are associ-
* Huang Jianjun ated with a life expectancy of approximately 2 years after
[email protected] diagnosis. Another type of tumor called meningiomas arises
from the meninges, where as cells growing abnormally in the
1
Guangdong Key Laboratory of Intelligent Information pituitary gland are referred to as pituitary tumors. Different
Processing, School of Electronics and Information
Engineering, Shenzhen University, Shenzhen, China types of brain tumors are depicted in Fig. 1. Many imag-
2 ing modalities such as magnetic resonance imaging (MRI),
Fujian Normal University, Fuzhou, China
computed tomography (CT), and positron emission tomog-
3
Imaging Department, Zunyi Medical University, Zunyi, raphy (PET) have been extensively used to examine brain
China

13
Vol.:(0123456789)
 E. U. Haq et al.

characteristic of images. Automatic and semi-automatic

segmentation techniques may be further divided into two
categories, including methods based on discriminative and
generative models (Wu et al. 2021). The former requires
low-level features of images to calculate probabilities for
solving multi-class detection problems, whereas the latter
requires prior information on brain structure and often fol-
low complex learning model. An approach based on Markov
random fields (MRF) (Agrawal et al. 2020) has been applied
in segmentation smoothing processes extracting useful
Fig. 1  Different types of brain tumors a glioma b meningioma c pitu- information from voxels within neighborhoods. Generative
itary models were shown to perform very well on unseen data;
however it has proven quite difficult to generate a suitable
probabilistic model from prior information. Many studies
tumors and other brain disorders. MRI has proven one of the have used MRF techniques for brain tumor segmentation,
most promising imaging techniques, and has been used for among which Menze et al. (2015) also obtained supervoxels
identification and treatment of cancer cells found in human from imaging data.
brains due to its ability to capture multiple parameters, qual- Discriminative models directly correlate low-level
ity contrast performance on soft tissues, and its ability to extracted features and a label for each given voxel. These
image scans in multiple directions. Different types of imag- low-level extracted features tend to be of several different
ing methods such is T1-weighted, T2-weighted and FLAIR varieties with varying characteristics, including local his-
(fluid attenuated inversion recovery) (Isin et al. 2016) may tograms (Brain Tumor Segmentation Challenge 2014), tex-
be obtained through MRI. These different types of imaging ture features (Subbanna et al. 2013, 2014), features based
scans obtained through MRI can be used to analyze informa- on region shape differences, and features based on align-
tion found in images and to study features of brain cancer ment with atlas and input raw data (Wang et al. 2019). Cer-
cells at a high level of detail. tain techniques such as support vectors machine (SVMs)
The precise and accurate segmentation process of brain (Bauer et al. 2013) and decision forests (Zikic et al. 2014)
cancer cells is an important and necessary task for appro- have been implemented as discriminative learning methods.
priate treatment scheduling and also to support the adop- Techniques based on random forest have achieved high accu-
tion of much-needed precautionary screening. However, the racy compared to other discriminative models approaches
identification of cancerous cells (i.e., necrosis, edema etc.) (Wang et al. 2018), as may be visually observed based on
in contrast to normal brain tissues remains a challenging the results of the MICCAI-BraTS Challenge from the years
task. Owing to complex variations in appearance, shape, 2012, 2013, and 2014. Currently existing generative based
location, and size of gliomas between patients and also models detect tumor tissue by considering it as a shape or
because the invasive nature of glioma cells within other tis- a signal diverging from the properties of normal brain tis-
sues distorts boundaries between them (Goetz et al. 2016), sue (Li et al. 2021). Prastawa (2004) proposed a generative
accurate segmentation remains a significant challenge. In based model for the identification of cancerous cells. Pos-
particular, manual segmentation is a tedious task and has terior probabilities of normal brain tissues were computed
been associated with numerous mistakes emerging from var- by aligning brain regions to an atlas, and cancerous regions
ious parameters. Thus, medical specialists use conventional were identified by finding posterior probabilities below a
approaches and rough calculations to analyze brain tumor set threshold value for such regions. Prastawa et al. (2003)
cells (Gryska et al. 2021). Hence, an efficient and precise obtained a probability map for abnormal brain tissues by
method is required for the automatic identification of brain aligning brain images to an atlas, and certain iteration tech-
tumors (Menze et al. 2015). niques were applied until a variation was observed in the
Many automated segmentation methods have been posterior probability from a certain threshold value. Many
proposed in the relevant literature, all of which depended other notable studies have been conducted such as (Khotan-
entirely on regions of interest, texture analysis, and histo- lou et al. 2009; Wu et al. 2020; Yang et al. 2021) depending
gram thresholds. However, these methods achieved only on symmetry and alignment features relative to the entire
relatively poor results in terms of accuracy. Among these, brain. However, aligning brain images with a certain contour
histogram thresholds are considered an effective method to has proven a challenging task. Nonetheless, numerous works
perform image segmentation tasks, but still involve some in the relevant literature have performed tumor segmentation
limitations. Threshold methods often fail to trace all parts and registration tasks simultaneously (MICCAI 2014; Pari-
separately when locating tissues and do not consider spatial sot et al. 2012). Deep learning techniques have demonstrated

13
An MRI‑based deep learning approach for efficient classification of brain tumors

the capability efficiently to extract increasingly complex fea- • We extend the GoogleNet architecture and present two
tures from input data (Bengio et al. 2013). Such techniques variant forms to obtain multi-scale features from MRI
aim to develop an effective method to automatically extract images for the identification and classification of brain
multiple features instead of using manual feature selection tumors. We introduce a cascaded pooling layer in the
in an image segmentation process (LeCun et al. 2015). In two GoogleNet variant architectures, which efficiently
the recent years, many approaches based on deep networks combines nearby features and decreases the computa-
have been implemented for medical image processing (Esti- tional complexity and load of the next step. We further
enne et al. 2020), object detection (Krizhevsky et al. 2017; apply a data augmentation concept to the brain tumor
Dieleman et al. 2015) and human recognition tasks (Peng classification problem, and a considerable improvement
et al. 2019). in performance is observed when data augmentation is
Some identification and classification problems have been combined with a deep learning model.
identified in the current literature. Methods and techniques • We thoroughly investigate the use of smaller 3 × 3 con-
used for segmentation, identification, and feature extraction volutional kernels in deep learning models to efficiently
based on machine learning, require significant computational stack convolutional layers while maintaining an equiva-
resources and associated effort, resulting in low overall sys- lent receptive field to larger kernels. The proposed deep
tem efficiency. Most prior machine learning techniques have learning models utilize multi-scale features for the detec-
not considered local dependencies of each label. Instead, tion and classification of brain tumor within 1.2–2.8 min,
these techniques simply focused on pixel classification. which is comparable with current state-of-the-art meth-
Thus, serious and challenging problems remain associated ods. The proposed method requires less time than prior
with these machine learning approaches. The efficiency and state-of-the-art techniques for the detection and classifi-
performance of existing techniques are insufficient consid- cation of brain tumors.
ering the medical implications of the issues and problems • Conditional random fields (CRFs) are applied as a post-
associated with classification and detection. Existing state- processing tool to appropriately resolve the incongruity
of-the-art methods rely on manual identification of tumor between segmentation accuracy and network depth.
regions prior to classification. Deep learning (DL) networks • The first proposed CNN Architecture used the Figshare
have the advantage of working on raw data and considering dataset to classify brain tumor as gliomas, meningiomas,
patches to accurately extract features. Convolutional neural or pituitary, and achieved an accuracy of 97.3% with a
networks have been considered as a suitable approach for dice similarity coefficient (DSC) of 95.8%, while the sec-
extracting favorable features to fulfill classification tasks. ond proposed CNN architecture achieved an accuracy of
Convolutional neural networks (CNNs) are based on a 96.5% with a DSC 94.3% using the BraTS 2018 dataset
shared-weight structure, minimizing their complexity, and to classify glioma grades as HGG or LGG.
have shown considerable promise compared to other con-
ventional machine learning methods in extracting complex We conducted a comprehensive comparative analysis in
hierarchal features. Owing to the wide application of deep terms of evaluation metrics of different convolutional neural
learning methods in solving real-time problems, techniques network architectures used for the detection and classifica-
for brain tumor segmentation (Pereira et al. 2016; Zhao et al. tion of brain tumors. The proposed network recorded better
2020; Thaha et al. 2019) based on deep learning models have detection and classification accuracy on the Figshare and
increasingly been proposed in recently published research. BraTS 2018 datasets.
To the best of our knowledge, no prior related work in the The remainder of this study is arranged as follows. The
relevant literature has considered data scarcity. The method related literature is presented in Sect. 2. Section 3 provides
proposed in the present work addresses the existing detection a description of the proposed method. Section 4 explains the
and classification problems evident in the literature. experimental setup used in this study and the results of the
The major contributions of this study are as follows. experiments. The proposed method is compared with state-
of-the-art algorithms in Sect. 5 and a discussion and ablation
• A robust and effective deep learning model is proposed study are presented in Sect. 6, and our final conclusions are
for the identification and classification of brain tumors. presented in Sect. 7.
• The design of the architecture of the proposed deep learn-
ing model applies two fast and systematic brain tumors
identification techniques based on deep CNNs for the 2 Related literature
detection and classification of different types of brain
tumors from magnetic resonance imaging data collected Most of the existing literature and recent studies on brain
from two publicly available datasets, including a Figshare tumor detection and classification generally involve image
brain tumor dataset and that of BraTS 2018. processing, extraction of valuable features, detection of the

13
 E. U. Haq et al.

extracted features, and classification of detected tumors into which were then used for a classification task after reducing
specific types. Many approaches based on deep learning and their dimensions.
convolutional neural networks have also been implemented Afshar et al. (2018) suggested a deep model based on
in the field of brain tumor segmentation to support proficient Capsule Neural Network for the efficient classification of
identification of cancer tissues. In Zhang et al. (2021), the brain tumors. They increased accuracy by modifying a fea-
authors introduced a hybrid cross-modality DL framework ture map in the convolutional layer of the Capsule network.
for the segmentation of brain tumors using multimodal MRI Abiwinanda et al. (2018) introduced a convolutional neu-
results, aiming to identify rich features within multimodality ral network for the identification and classification of brain
data to compensate for insufficient data volume. The authors tumors using five architectures. Their proposed architec-
of Zhou et al. (2021) presented an effective 3-dimensional ture achieved the best accuracy among methods compared
residual neural network for the efficient segmentation of by combining two convolutional layers, a ReLU layer and a
brain tumors. Their proposed model offered viability, lower max-pooling layer. Anaraki et al. (2019) proposed a network
GPU memory usage, and less complexity. In Khan et al. architecture using a genetic algorithm; however, this method
(2020), the authors presented a multimodal hybrid classifi- did not obtain promising results on brain tumor classification
cation approach based on DL to classify brain tumor types. tasks. Paul et al. (2017) presented a convolutional neural net-
The proposed approach utilized a fully connected (FC) layer work for brain tumor classification with two convolutional
for the extraction of features and used BraTS 2015, BraTS layers, each with 64 kernels, and four dropout layers. Erto-
2017, and BraTS 2018 for experimental purposes. sun and Rubin (2015) used two classifiers to classify brain
The authors in Zikic et al. (2014) introduced a shallow tumors; however, they achieved only poor results despite
CNN in which a max-pooling layer divided every two neigh- using pathological images to train their proposed network.
boring convolutional layers, and the entire network was pre- Soltaninejad et al. (2018) proposed a three-dimensional
ceded by a fully connected layer and a softmax layer. The super voxel-based learning technique to accurately perform
work done in Urban et al. (2014) assessed the effective use brain tumor segmentation using traditional MRI and diffu-
of 3-dimensional filter in the segmentation process; how- sion tensor imaging (DTI) brain images. A multimodal MRI
ever, most studies in the literature have used 2-dimensional dataset was used to generate the supervoxels. Then, features
filters. 3-dimensional filters outperformed other types in were extracted for each supervoxel. The extracted features
capturing the 3-dimensionalcharacteristics of images, but were used as input to a random forest algorithm, which clas-
were limited by increased computational complexity. Binary sified them as tumor core, edema, or normal brain tissue. A
convolutional neural networks were introduced by Lyksborg clinical dataset and the BraTS 2013 dataset were used to
et al. (2015) for the complete identification of brain tumors, evaluate the performance of their proposed method, which
and then used a multi-class CNN model to distinguish the achieved 86% detection sensitivity and 0.84 dice score on the
sub-regions of brain tumors. clinical dataset; on the BraTS 2013 dataset, it attained 96%
Dvorak and Menze (2015) presented a useful approach and 0.89 detection sensitivity and dice score, respectively.
based on CNNs as a learning method for segmentation of Dong et al. (2017) proposed aU-net-based deep convolu-
brain tumors based on partitioning different regions of brain tional neural network as an automated brain tumor segmen-
tumors into binary sub-tasks. The authors in Kamnitsas et al. tation technique. Their proposed model was evaluated using
(2017) presented an approach using two patches of different the BraTS 2015 dataset, and achieved satisfactory results on
sizes to extract multi-scale features. In Havaei et al. (2017), the task of classifying HGGs, LGGs, and combined cases.
the authors proposed a cascaded CNN network for multi- Soltaninejad et al. (2016) proposed a superpixel-based clas-
scale feature extraction. Furthermore, they solved the imbal- sification method for abnormal brain tissues using FLAIR
ance problems of label distribution by using a two-phase MRI data. Different image features such as intensity, Gabor
training approach. Pereira et al. (2016) presented multiple textons, and fractal analysis were calculated from the data
deep CNN frameworks for the segmentation of glioma sam- of each brain region to perform classification efficiently.
ples at different levels. The extraction of multi-scale features Two classifiers, extremely randomized trees (ERT) and
by CNNs has a wide range of applicability and significant support vector machine (SVM), classified each superpixel
advantages in segmentation tasks (Shelhamer et al. 2017). as timorous or non-tumorous. The results of both classifi-
Hu et al. (2018) presented a multi-scale approach to the seg- ers were then compared. Two datasets, including a clinical
mentation of retinal vessels based on a CNN. In the method dataset comprising 19 FLAIR MRI images and the BraTS
proposed in Hu et al. (2018), all multi-scale images obtained 2012 dataset, were used to evaluate the performance of their
in different layers of a convolutional neural network were proposed method. The ERT classifier achieved the highest
fused to obtain a probability map. Studies conducted by evaluation indices for both the clinical and the BraTS 2012
Zacharaki et al. (2009) and Cheng et al. (2015) used feature datasets. For the clinical dataset, the ERT classifier attained
engineering to effectively perform extraction of features, 89.48%, 6%, and 0.91 detection sensitivity, balanced error

13
An MRI‑based deep learning approach for efficient classification of brain tumors

rate, and dice score, respectively, while the same evaluation domain and transform domain filtering. Wiener filters and
indices on the BraTS 2012 dataset were 88.09%, 6%, and median filters are assumed to minimize the root mean square
0.88, respectively. error between the original and output images. Transform
In Rajinikanth et al. (2017), the authors suggested a meta- domain filtering primarily includes Fourier transform and
heuristic optimization-based approach for analyzing brain wavelet transforms methods. Transform domain filtering
tumor MRI. The proposed method was used to enhance method aim to decompose noise and original signals into
and remove core and edema regions of brain MRI in the small coefficients, which allows noise to be eliminated rela-
first stage. The proposed model was tested and validated tively easily with low frequency coefficients. In Dey et al.
on FLAIR, T1C, and T2 MRI modalities. In Rajinikanth (2015), a filtering approach based on a genetic algorithm
et al. (2018), the authors presented a hybrid technique based was implemented for parameter optimization in brain MRI
on a discrete wavelet transform and principal component denoising. A multi-threshold approach (Pavidraa et al. 2020)
analysis to predict irregularities in MR images. Their pro- was utilized based on social group optimization (SGO)
posed approach was validated on the BraTS 2013 dataset. and Tsallis entropy (TE) to analyze and examine MRI
The authors of Satapathy and Rajinikanth (2018) suggested images and to improve the segmentation process. In addi-
a hybrid machine learning-based approach for predict- tion, a Level Set Segmentation (LSS) method was utilized
ing anomalies in FLAIR and T2 MRI modalities using the to extract a tumor region from threshold MRI images. An
Jaya algorithm and Otsu’s method. Their proposed method image processing-based approach was proposed to examine
assessed brain tumors in two-dimensional MRI slices in ischemic stroke lesions in MRI using FLAIR and weighted
axial, sagittal, and coronal views. fusion modalities. Their proposed method was divided into
two steps, including pre-processing based on fuzzy Tsallis
entropy and post-processing based on segmentation methods
3 Proposed method used to extract features via image-wise supervised learning
(ISL) from pre-processed images (Raja et al. 2018). A hybrid
Figure 2 shows the proposed method, which consists of three method focused on fuzzy and brainstorm-optimization tech-
main stages performing pre-processing, classification via niques was used for the segmentation and detection of brain
CNN, and post-processing based on CRFs. tumors in MRI images. Their proposed method was tested
and validated on the BraTS 2018 dataset, and it was found
3.1 Pre‑processing to be effective and stable (Narmatha et al. 2020).A support-
based approach was used to perform efficient identification
Magnetic resonance images are substantially contaminated of medical renal diseases, considering only relatively small
by the three most common types of noise, including Gauss- image features. Their proposed approach used ultrasound
ian, salt and pepper, and speckle noise. Many denoising and images, and achieved a classification accuracy of 85.7%
filtering approaches have been adopted to eliminate these (Sharma and Virmani 2017).
types of noise from MRI images. Denoising methods are The quality and intensity of MRI images are always
commonly divided into two main classes, including spatial altered by bias field distortion. Owing to this distortion, the

Fig. 2  Overview of the proposed method

13
 E. U. Haq et al.

quality and intensity of the same tissues vary across multi- 4. Mean intensity value and standard deviation are calcu-
ple images. The N4ITK method has been applied to ensure lated for all pixels in the image.
equivalent levels of intensity distribution of tissues across 5. The parameters of the proposed model are initialized
different objects in MRI sequence images; however, this has after completing the N4ITK and Nyul et al. (2000) nor-
proven insufficient. In fact, intensity levels still vary even malization method.
for a given scanner and a given patient over varying time 6. The value of each parameter is updated.
periods. The intensity normalization approach proposed by
Nyul et al. (2000) was used to ensure equivalent intensity 3.2 Region proposal network
and contrast levels of tissues across all MRI images for the
same patients across varying time intervals. A set of inten- A region proposal network (RPN) is used to produce vari-
sity pointers were recognized for each MRI sequence from able-scale rectangular object proposals for the detection of
training set in this intensity normalization approach. After various types of brain tumors. The applied RPN operates
successful training, the original intensities between two on the basis of anchor boxes, and comprises two leading
pointers were linearly transformed to obtain a correspond- constraints, including a classifier and a regressor. The prob-
ing learned pointer. The mean intensity value and standard ability of a proposed region representing the target object is
deviation of all pixels were calculated for each pixel in the calculated by the classifier. The dimensions of the proposals
image. are derived through regression. The generated rectangular
Consider Zij (v) to represent the image intensity at each object region proposals are further used as input for a deep
voxel using a specific imaging modality. Normalization, CNN model to perform classification and adjustment of rec-
then, is a process of alteration from Zij (v) to Nij {Zij (v)}. The tangular bounding boxes. The deep CNN model classifies
histogram of intensities Zij (v) in an image represents a mix- the bounding boxes into one of the three brain tumor types
ture of different densities, which can be shown through the and adjusts each proposal with respect to tumor size and
following equation. region.
K
∑ 3.3 Proposed architecture
gij (x) = qijk gijk (x). (1)
k=1
In this study, our objective is to develop an appropriate
gijk (x) represents the intensity densities of empty spaces or model for identifying and classifying brain tumors. Because
known tissues, whereas qijk represents the relative weights tumor anatomical structures present considerable variabil-
of components varying from k = 1,2, ..K . The distribution ity, effective segmentation processes remain challenging.
function gijk (x) can be estimated for all components from To overcome this issue, a deep convolutional neural net-
k = 1,2, ..K and for all values of i and j , which in turn work is designed and intensity normalization is tuned for
results in a normalized estimator having zero mean and unit each tumor type, because the dataset used for the evalua-
variance. tion included images of gliomas, meningiomas and pitui-
( ) tary tumors, and the appearance and image patterns of the
fijk (x) = 𝜎ijk gijk 𝜇ijk + 𝜎ijk x , (2)
three types of tumors differ significantly from one another.
where 𝜇ijk and 𝜎ijk respectively represent the mean and stand- The pooling layer is essential for achieving invariance and
ard deviation of gijk (x). minimizing unnecessary details; however, removing essen-
In the training phase of the proposed method, the mean tial details may have a negative impact. All layers used the
and standard deviation are determined for all extracted fea- LReLu as an activation function, except the last layer, which
tures after the patches have been normalized, our key aim used a softmax activation function. Dropout was only used
being to normalize all features to equivalent means and in FC layers.
variances. The mean and variance measured in the training The architecture of the two proposed deep convolutional
phase are then used to normalize the features in the testing neural networks is based on GoogLeNet. We used two con-
phase. The following steps are involved in pre-processing figurations of the basic GoogLeNet (Szegedy et al. 2015)
procedure. developed by Szegedy in this work. The first architecture
contains two convolutional layers, four cascaded max-
1. Patches and labels of each image are given at input. pooling layers, nine inception models, one FC layer, and
2. The N4ITK method is applied to ensure equivalent levels the last linear layer with softmax as activation in the out-
of intensity distribution of tissues for each input. put, is used for the identification and classification of three
3. Nyul et al. (2000) normalization approach was applied to types of brain tumors, namely glioma, meningioma, and
attain equivalent intensity and contrast levels of tissues. pituitary. The later configuration of GoogLeNet architecture
contains four convolutional neural networks, four cascaded

13
An MRI‑based deep learning approach for efficient classification of brain tumors

max-pooling layers, nine inception models, one FC layer, classification of gliomas, meningiomas, and pituitary tumors
and a final linear layer with a softmax activation function includes three outputs to adapt to the target domain, whereas
produces the output. The main aim of this configuration of in the CNN module used for the classification of HGG and
GoogLeNet is to classify grades of glioma as. HGG or LGG. LGG tumors, a fully connected layer was inserted with an
The model used for glioma grade detection is deeper than output size of two. Tables 1 and 2 present the structures
the model used for detection of gliomas, meningiomas, and and parameters of the first and second CNN architectures,
pituitary tumors because going deeper in the latter case did respectively.
not improve the results; rather, it primarily increased com- The following concepts are significant to the operation of
putational complexity and over-fitting. Dropout was set to the proposed model.
p = 0.5for LGG and p = 0.1 for HGG, because the dataset
used for evaluation included more HGG than LGG cases. 3.3.1 Xavier Initialization
Additionally, the appearance and patterns of HGG and LGG
differ significantly from one another. Xavier initialization (Xavier Glorot 2010) is used in the pro-
The first GoogLeNet variant model was used to classify posed CNN model to achieve a fast and stable convergence.
brain tumors as gliomas, meningiomas, or pituitary tumors, Xavier initialization maintains the activation function and
while the second variant configuration of GoogLeNet was gradient within control levels; otherwise a back propagated
used to classify the grade of gliomas as HGG or LGG. gradient may cause explosion or itself vanish. Xavier ini-
The first and second CNN models are illustrated in Figs. 3 tialization enables a signal or different features of an image
and 4, respectively. The first deep architecture describes a can be processed deeper within a network. If a signal or an
CNN model for the classification of gliomas, meningiomas, image’s features are initiated with excessively small weights,
and pituitary tumors, while the second deep architecture the signal or features will progressively shrink as they move
embodies a CNN structure for the classification of HGG through each layer, and gradually become too small to be
and LGG occurrences. The CNN architecture used for the useful. Similarly, if a signal or an image’s features are

Fig. 3  The proposed model (1st

deep architecture) used in this
work for the identification and
classification of Glioma, Menin-
gioma and Pituitary

Fig. 4  The proposed model

(2nd deep architecture) used in
this work for the identification
and classification of HGG and
LGG

13
 E. U. Haq et al.

Table 1  Architecture of the CNN structure used for the classification initiated with excessively high weight, the signal becomes
of glioma, meningioma and pituitary tumor too large to be usable as it moves through each layer. Xavier
Type Filter size Stride Input size # Filters initialization ensures that the weights are initiated correctly,
holding the signal within an acceptable range of values as it
Image input 448 × 448 –
passes through several layers.
Conv. layer 3×3 1×1 4 × 224 × 224 64
Max-pooling layer 3×3 2×2 64 × 224 × 224 –
3.3.2 Convolutional layer
Conv. layer 3×3 1×1 64 × 112 × 112 64
Max-pooling layer 3×3 2×2 64 × 112 × 112 –
CNN have been used to achieve some outstanding and break-
Inception 3a – – 128 × 56 × 56 128
through results in a wide variety of fields. The convolutional
Inception 3b – – 128 × 56 × 56 128
layers in CNN are used to learn features through convolving
Max-pooling layer 3×3 2×2 128 × 56 × 56 –
kernels across input image or signal. Therefore, the weight
Inception 4a – – 128 × 28 × 28 128
of kernel is a special parameter representing the connection
Inception 4b – – 128 × 28 × 28 128
between a unit in a feature map and the previous layer. The
Inception 4c – – 128 × 28 × 28 128
weight of a kernel is a special parameter connecting each
Inception 4d – – 256 × 28 × 28 256
unit in a feature map to the previous layer. Moreover, kernel
Inception 4e – 1×1 256 × 28 × 28 256
weights also enhance and maximize certain characteristics
Max-pooling layer 3×3 2×2 256 × 28 × 28 –
and parameters of the input. Back-propagation phenomenon
Inception 5a – – 256 × 14 × 14 256
is used in CNN training phases to adjust the weight of ker-
Inception 5b – – 256 × 14 × 14 256
nels to boost certain characteristics of an input signal or
Avg-pooling layer 3×3 2×2 256 × 14 × 14 –
image. Adapted kernel weights in training phases are also
Fully connected layer – – 480 × 7 × 7 –
used in the corresponding testing phase to obtain feature
Softmax layer – – 480 × 7 × 7 –
maps for each image. In the case of our proposed method,
as the entire unit in a given feature map equally shares the
kernels, convolutional neural networks are less exposed
to issues of over-fitting and much easier to train. Though
using kernels, neighborhood information is also considered
to retain properties of the source information. The output of
Table 2  Architecture of the CNN structure used for the classification every neural unit's output is subjected to a nonlinear activa-
of HGG and LGG glioma grade tion function. The features are enhanced in the first convo-
Type Filter size Stride Input size # Filters lutional layers.

Image input 448 × 448 –

3.3.3 Pooling layer
Conv. layer 3×3 1×1 4 × 224 × 224 64
Conv. layer 3×3 1×1 64 × 224 × 224 64
Pooling is used to combine nearby features, which causes the
Conv. layer 3×3 1×1 64 × 224 × 224 64
representation to become more compact and invariant, and
Max-pooling layer 3×3 2×2 64 × 224 × 224 –
thus decreases the computational complexity and load in the
Conv. layer 3×3 1×1 64 × 112 × 112 64
next steps. Max-pooling or average-pooling layer are con-
Max-pooling layer 3×3 2×2 64 × 112 × 112 –
ventionally used to efficiently combine extracted features.
Inception 3a – – 128 × 56 × 56 128
Our proposed model includes four cascaded pooling layers.
Inception 3b – – 128 × 56 × 56 128
Max-pooling layer 3×3 2×2 128 × 56 × 56 –
3.3.4 Data augmentation
Inception 4a – – 128 × 28 × 28 128
Inception 4b – – 128 × 28 × 28 128
Data augmentation or oversampling techniques are used to
Inception 4c – – 128 × 28 × 28 128
increase the size of the available data and minimize over-
Inception 4d – – 256 × 28 × 28 256
fitting. To increase the number of images in the CNN train-
Inception 4e – 1×1 256 × 28 × 28 256
ing dataset, we investigated data augmentation techniques
Max-pooling layer 3×3 2×2 256 × 28 × 28 –
such as flipping, rotation, and translation. This is also more
Inception 5a – – 256 × 14 × 14 256
precisely referred to as class balancing or oversampling
Inception 5b – – 256 × 14 × 14 256
rather than augmentation. Input images are rotated at vari-
Avg-pooling layer 3×3 2×2 256 × 14 × 14 –
ous angles, such as 90°, 180°, and 270°, in rotation-based
Fully connected layer – – 480 × 7 × 7 –
data augmentation, while they are reflected in horizontal
Softmax layer – – 480 × 7 × 7 –
and vertical directions in flip augmentation. Some studies in

13
An MRI‑based deep learning approach for efficient classification of brain tumors

the relevant literature (Maas et al. 2013) have also consid- network with probability p in each training step. This tech-
ered image translations; however, this could yield incorrect nique enables the network nodes to better represent the data
results on the segmentation process. Data augmentation is and regularization prevents the nodes from co-adapt to each
important because it produces virtual data from an actual other. Moreover, all nodes were used in the testing phase.
dataset. The synthetic minority oversampling technique was
adopted as a data augmentation method in this study to deal 3.3.7 Loss function
with the class imbalance problem. This technique uses a
linear mapping of feature vectors to generate new data points In this study, samples weights are used as a simple and
based on current minority class data points. effective strategy to handle class imbalance problems. The
The results of this data augmentation results may be key idea is to weigh the loss for various samples as either
observed in Fig. 5. belonging to a major or minor class. In this method, a higher
weight is assigned to the loss experienced by the samples of
3.3.5 Activation function minor class. During the training process, the main aim is to
minimize the loss function. In our proposed approach, we
Activation functions are commonly used to transform data apply a categorical cross-entropy loss function, which can
nonlinearly. Leaky rectified linear units (LReLu) were used be expressed as
in this work, which achieved better results and sped up the ∑ ∑ ( )
training process compared to other activation functions such H=− cj,k log ĉ j,k ,
(3)
as hyperbolic tangent functions. We considered LReLu suit- j∈voxels k∈classes

able as an activation function, because it has low compu- c represent the probabilistic predictions.
where ̂
tational complexity without saturation, and does not cause
a vanishing gradient problem. It also eliminates the dying 3.4 Post‑processing based on CRFs
ReLu problem, as no node is expected to die in LReLu acti-
vation functions. The most important feature of LReLu is The proposed method considers local dependencies in MRI
that it does not exclude or eliminate the effect of negative images using a cascading pooling layer; however, in some
values, and give a small slope rather than zero slope for cases this is insufficient for medical image processing.
negative values. A softmax activation function is used in the Owing to various complexities in the relevant anatomical
last FC layer of the proposed model. structures, such modeling remains a highly challenging task.
Furthermore, temporal and spatial MRI data must be con-
3.3.6 Regularization sidered in the proposed approach. Therefore, the probability
map obtained using the proposed deep CNN model can be
In regularization, dropout is used in the fully connected (FC) further refined. Many applications of CRFs pertaining to
layers to remove over-fitting. Nodes are removed from the

Fig. 5  Data augmentation

applied on glioma image a
original image b 90° rotation c
180° rotation d 270° rotation e
horizontal flip f vertical flip

13
 E. U. Haq et al.

medical image processing have been proposed in recently dataset (Cheng 2017), first developed by Cheng and, con-
conducted research, as they are considered suitable for mod- taining brain tumor images. We used the Figshare (Cheng
eling spatial data dependencies. In the case of brain tumors, 2017) dataset to evaluate and analyze the performance of
CRFs are important in modeling the relationship between our proposed deep CNN model. This dataset consisted of
image pixel features and their associated labels, as well as a total of 3064 T1-weighted contrast-enhanced brain MRI
the relationship between neighboring image pixel features images collected from 233 patients. The three main types
and their labels. The CRF training process used by Zhao of brain tumor MRI images available in this dataset were
et al. (2018) was computationally complex and expensive. In gliomas, meningiomas, and pituitary tumors. Among the
this study, we use a CRF method as a tool to further improve 3064 images, 1426 were glioma images, 708 showed men-
the accuracy of our proposed deep CNN model and offer ingiomas, and 930 showed pituitary tumors. The Figshare
relatively low computational complexity. dataset is available in MATLAB MAT-file (.mat) format for
Considering a 2-dimensional image X, the pixel public use on the Figshare website. Each type of tumor in
information of the image X can be represented the MAT-file is represented by a patient ID and unique label.
by Xi where i = 1,2, 3, … , N, its label is given by The images in this dataset are presented in axial, coronal,
Zl where l = 1,2, 3, … , C , and the image segmentation prob- and sagittal views. In the experimental work, each image was
lem is represented as an optimization problem. The energy given as input to the proposed deep convolutional model.
function of the CRFs is minimized to efficiently train the Our proposed model was also tested and evaluated using
model. the BraTS 2018 dataset, which contains 285 total glioma
N N
images available from four co-registered MRI modalities
∑ ( ) ∑ ( ) (T1, T1C, T2, and FLAIR). These 285 glioma cases com-
E(Z) = 𝜓u zi + 𝜓p zi , zj , (4)
i i,j prised 210 HGG and 75 LGG images. The dataset also con-
tained 66 patient images of unknown grades.
where u, p are segmentation labels,( and
) i, j are pixels of
image X, and the unary potential 𝜓u zi represents a nega- 4.2 Implementation details
tive log-likelihood function.
( ) ( ) To validate the feasibility and compatibility of the proposed
𝜓u zi = −logP zi |X , (5) model, we conducted several comparative studies on the
where P(zi |X) represents the probability obtained by the pro- Figshare and BraTS 2018 datasets. In the first phase, we
posed deep CNN for a pixel i . The pairwise potential can conducted a series of experiments to evaluate the classifica-
be expressed by the following equation, which is used to tion efficiency of the two proposed deep models. Second,
measure influence between any two pixels. we validate the proposed framework by comparing the clas-
sification performance of the deep CNN model with and
( ) ( M
)∑ ( ) without the CRFs. We carried out extensive experiments
𝜓p zi , zj = 𝛼 zi , zj wm km gi gj . (6) to analyze the effectiveness of the proposed method on the
m=1
T1-weighted contrast-enhanced modality of the Figshare
Here, M represents the total number of Gaussian kernels, dataset using the proposed first CNN model and four co-
and registered MRI modalities (T1, T1C, T2, and FLAIR) of
( w )represents the weight of the m − th1Gaussian kernel,
m

𝛼 zi , zj are label compatibility functions,k is an appearance the BraTS 2018 dataset using the second CNN model. In the
kernel, which assigns the same class number to neighbor- experiments, we divided the training process into two stages.
hood pixels having the same intensity, and k2 is a smoothness A deep CNN model was used for the classification task, and
kernel removing small isolated regions. We use a threshold a probability map was obtained. In the second stage, the
value to exclude clusters obtained by the deep CNN model obtained probability maps were fed into the CRF network
that are incorrectly labeled as tumors. to achieve more accurate classification results.
We included a total of 3349 images from the Figshare
and BraTS 2018 datasets (3064 Fishare + 285 BraTS 2018)
4 Experimental setup and results for use in the training and testing phases of the proposed
deep CNN model. There were a total of 2609 images in the
4.1 Dataset training dataset, comprising 2408 images from FigShare and
201 from BRATS 2018, while the testing dataset contained
We used two publicly available datasets, namely Figshare 740 images with 656 images for FigShare and 84 for BRATS
and BraTS 2018 to validate and evaluate our proposed 2018.
model. The first dataset was a publicly available Figshare For testing, 282 images of gliomas, 173 of meningiomas,
and 201 of pituitary tumor samples were chosen from the

13
An MRI‑based deep learning approach for efficient classification of brain tumors

Table 3  Training and testing details faster performance compared to the batch method as a result
Dataset Total Training Testing of using a single update in the training process. We used 100
epochs to train the proposed deep model with 1000 iterations
Figshare 3064 2408 656 for each epoch. 14 h were required to complete the training
BRATS 2018 285 201 84 process on an NVIDIA CUDA GPU with multiple proces-
Total 3349 2609 740 sors of 2.80 GHz, a 1 TereByte HD, and a 128 Gigabyte
SSD. After the completion of 100 epochs, the proposed deep
CNN model achieved satisfactory training accuracy.
Table 4  Training and testing details of Figshare dataset
Dataset Total Training Testing
4.3 Results

Figshare Glioma 1426 1144 282 The experimental results clearly demonstrate that increas-
Meningioma 708 535 173 ing the system structure and complexity improved the per-
Pituitary 930 729 201 formance effectiveness of the proposed model. Each tumor
image belonging to a given class differs significantly from
other tumor image of same class in terms of texture and
Table 5  Training and testing details of BraTS2018 dataset intensity. The proposed two architectures are able to detect
Dataset Total Training Testing
and classify the type of brain tumors irrespective of their
varying appearance and contrast. Figure 6 shows different
BRATS 2018 HGG 210 149 61 brain tumors which vary in shape, types, and location. Fig-
LGG 75 52 23 ure 7 shows axial, coronal and sagittal views of pituitary
tumors included in the dataset. As shown in Fig. 8, we dis-
tinguish HGG and LGG glioma grades.
FigShare dataset, while 61 images of HGGs and 23 of LGGs In this study, we compared the performance of the two
were chosen from BraTS 2018. Brain tumors, namely glio- proposed architecture models, as well as the performance of
mas, meningiomas, pituitary tumors, and grades of glioma both models with and without using CRFs on the Figshare
(HGG and LGG) were detected and classified from the test and BraTS 2018 datasets. Table 6 shows accuracy metrics
dataset. Tables 3, 4, and 5 show details of the training and for the two models in terms of sensitivity, specificity, PPV,
testing phases conducted experimentally using the Figshare NPV, accuracy, and DSC. Figure 9 shows a graphical rep-
and BraTS 2018 datasets. resentation of the obtained results in terms of sensitivity,
The two main issues associated with batch gradient or specificity, PPV, NPV, accuracy, and DSC. From Table 7, it
conventional optimizer methods are that they cause over- may be observed that the results obtained by the proposed
fitting and slow convergence for large datasets) and do not model with CRFs were slightly higher than those without.
facilitate new data in the training process. Stochastic gradi- Therefore, we can conclude that post-segmentation through
ent descent (SGD) methods have been extensively used to CRFs achieved a higher performance. Figure 10 shows the
overcome such issues and facilitate training processes after detected and classified results, showing gliomas, meningi-
the network learns only few or small training features. The omas, and pituitary tumors based on the first architecture.
proposed deep CNN models were trained using an SGD Figure 11 shows the detected and classified results on HGG
optimizer with a learning rate of 0.01 and a momentum of and LGG grade gliomas based on the second architecture.
0.8. A performance comparison between batch gradient Efficient and correct segmentation of various sub-regions
descent (BGD), and SGD evaluated that SGD method had of brain tumors such as peritumoral edemas and enhancing

Fig. 6  Brain tumors a Glioma b

Meningioma c Pituitary

13
 E. U. Haq et al.

Fig. 7  Pituitary tumors a axial b

Coronal c Sagittal

Fig. 8  Glioma grade a HGG b LGG

versus non-enhancing tumors plays a key role in the diag-

nosis and treatment of brain tumors. However, automated
Fig. 9  Graphical representation of obtained results in terms of sensi-
detection, classification and segmentation of the different
tivity, specificity, PPV, NPV, Accuracy and DSC
sub-regions of brain tumors remain a challenging task. The

Table 6  Accuracy metrics in terms of sensitivity, specificity, PPV, NPV, accuracy and DSC for the two models
Method Metrics
Tumor type TP TN FP FN Sensitivity Specificity PPV NPV Accuracy (%) DSC (%)

Proposed 1st CNN Glioma 271 365 09 11 0.960 0.975 0.967 0.970 96.9 96.4
Architecture used for Meningioma 167 473 09 06 0.965 0.979 0.943 0.987 97.7 95.7
Figshare dataset
Pituitary 191 447 08 10 0.950 0.982 0.959 0.978 97.2 95.5
Proposed 2nd CNN HGG 59 52 3 2 0.967 0.945 0.951 0.962 95.6 95.9
Architecture used for LGG 22 91 2 1 0.956 0.978 0.916 0.989 97.4 93.6
BRATS 2018 dataset
Combined 29 83 1 3 0.906 0.988 0.966 0.965 96.5 93.5

observed that the curve first begins to drop sharply; however,

proposed network was also utilized to conduct experiments
as a result of the small batch size, some fluctuations are evi-
on segmentation of different sub-regions of brain tumors.
dent. These fluctuations were reduced almost to zero after
Figure 12 shows experimental results on detection tasks for
16 k iterations and tend to disappear.
different brain tumor sub regions.
Figure 14 illustrates the accuracy and loss during the
Figure 13 depicts the accuracy and loss in a graphical
training phase using the second architecture. The accuracy
representation of the training process of the first architec-
graph in Fig. 14 shows that the proposed method achieved
ture. The accuracy graph in Fig. 13 shows that the proposed
more than 90% accuracy after 270 iterations and improved
method achieved an accuracy of almost 90% after 15 k itera-
overall accuracy to 91.7% achieved during the test phase.
tions, while from the loss graph in Fig. 13, it can be clearly

13
An MRI‑based deep learning approach for efficient classification of brain tumors

Table 7  Accuracy metrics in terms of sensitivity, specificity, PPV, NPV, accuracy and DSC for the two models with and without CRFs
Methods Sensitivity Specificity PPV NPV Accuracy (%) DSC (%)

1st CNN Architecture used for Figshare dataset 0.934 0.945 0.948 0.957 93.7 91.5
2nd CNN Architecture used for BRATS 2018 dataset 0.925 0.947 0.939 0.953 95.4 93.2
1st CNN Architecture used for Figshare dataset + CRFs 0.958 0.979 0.960 0.978 97.3 95.8
2nd CNN Architecture used for BRATS 2018 dataset + CRFs 0.948 0.974 0.945 0.972 96.5 94.3

Fig. 10  Detection and Classi-

fication result showing glioma,
meningioma and pituitary from
left to right

Specificity can be expressed as a model’s ability correctly

to identify and diagnose an actual brain tumor.
tn
Specificity = × 100%.
tn + fp

Sensitivity describes a model’s capability accurately to

identify relevant brain tumors, and it can be represented as
follows.
tp
Sensitivity = × 100%.
Fig. 11  Detection and Classification result showing HGG and LGG tp + fn
from left to right
PPV (positive predicted values) is identical to precision.
It computes true positive measurements, which can be cal-
The loss graph in Fig. 14 shows that the loss dropped sharply culated using the expression below.
and finally disappeared after 190 iterations and tends toward
zero. PPV =
tp
× 100%.
The first CNN architecture used for the Figshare data- tp + fp
set to classify brain tumors as gliomas, meningiomas, or
NPV (Negative predicted values) shows the probability of
pituitary tumors, achieving an accuracy of 97.3%, while the
the absence of a brain tumor based on true negative values,
second CNN architecture achieved an accuracy of 96.5%,
and can be expressed as follows.
using the BraTS 2018 dataset to classify glioma grades as
HGG or LGG. Figure 15 shows the confusion matrix for tn
NPV = × 100%.
both architectures. tn + fn

4.4 Evaluation metrics Accuracy specifies the system’s capability accurately to

discriminate between various types of brain tumors. Accu-
True negative (tn), true positive (tp), false positive (fp), racy was determined using the following formula.
and false negative (fn) are four significant and important tp + tn
outcomes conventionally used to evaluate the performance Accuracy = × 100%.
tp + tn + fn + fp
of classification methods. The performance criteria for the
proposed methods were based on these four outcomes, which The Dice similarity coefficient (DSC) is a performance
were also used to evaluate its effectiveness; its feasibility and indicator used to measure the overlap between samples, and
efficiency were assessed using the indices below. can be expressed as follows.

13
 E. U. Haq et al.

Fig. 12  Results of detection of different sub-regions of brain tumors edema, green enhancing tumor, cyan non-enhancing tumor d blue
a blue represents edema, orange enhancing tumor, yellow necro- represents edema, cyan enhancing tumor, orange non enhancing
sis, green non-enhancing tumor b cyan represents edema, yel- tumor e cyan enhanced tumor, green edema, yellow non-enhancing
low enhanced tumor, green non-enhancing tumor c blue represents tumor

Fig. 13  Graphical representa-

tion of accuracy and loss over
training iterations on glioma,
meningioma and pituitary tumor
identification tasks

Fig. 14  Graphical representa-

tion of accuracy and loss over
the training iterations on HGG
and LGG detection tasks

2tp recall values were calculated from the tabulated results for
DSC = × 100%.
2tp + fp + fn each tumor type. Recall can be expressed by the following
equation.
The mean average precision is an important parameter
for evaluating the performance of the proposed model. The TP
Recall = .
obtained results were recorded on an Excel spreadsheet in Total number of boxes predicted
order to calculate the mean average precision. Precision and

13
An MRI‑based deep learning approach for efficient classification of brain tumors

Fig. 15  Confusion matrix of the proposed study a Proposed 1st CNN Architecture used for FigShare dataset b Proposed 2nd CNN Architecture
used for BraTS 2018 dataset

Fig. 17  AUC ROC curves of different tumor class

Fig. 16  Graphical representation of Precision vs. Recall curves

4.5 Comparative analysis of the proposed two

empirical architectures
Precision vs. recall curves for gliomas, meningiomas, and
pituitary tumors are shown in Fig. 16. From the precision In this section, the inception layers involved in the two pro-
vs. recall curves, it can be observed that 13 recall points posed architectures of deep CNN are investigated. Inception
were selected between 0 and 1.2, with a step size of 0.1.
The precision values were then used to calculate the average
precision, which can be calculated from following equation. Table 8  Experimental results of 1st Deep model on test set of brain
tumor dataset—Figshare
1
AveragePrecision = (Precision(Recall = 0) Layer (Output) Sensitivity Specificity Precision Accuracy
13
+ … + Precision(Recall = 0) Inception 3a 56.87 57.40 48.97 48.35
Inception 3b 68.18 75.02 76.12 58.02
Inception 4a 79.22 78.85 88.93 83.13
Figure 17 shows the area under the receiver operating Inception 4b 94.54 89.22 94.52 95.99
characteristic (AUC ROC) curves for different tumor classes, Inception 4c 93.48 91.09 93.30 95.87
i.e., gliomas, meningiomas, and pituitary tumors, as well as Inception 4d 97.69 95.67 96.70 94.90
HGG, and LGG. Inception 4e 97.46 95.90 96.50 95.87
Inception 5a 92.24 94.57 94.09 96.60
Inception 5b 96.77 96.18 96.50 97.86

13
 E. U. Haq et al.

Table 9  Experimental results of 2nd Deep model on test set of brain best prediction results among the methods compared, which
tumor dataset—BRATS 2018 demonstrates the reliability of our proposed model. For the
Layer (Output) Sensitivity Specificity Precision Accuracy segmentation of brain tumors, the authors in Hu et al. (2019)
used a multi-cascaded convolutional neural network com-
Inception 3a 53.37 55.74 46.84 48.72
bined with a conditional random field. The multi-cascaded
Inception 3b 65.71 72.34 73.46 57.98
CNN was proposed for multi-scale feature extraction. Use
Inception 4a 76.12 77.98 85.12 82.76
of a stacking CNN simply improves the segmentation per-
Inception 4b 92.70 87.12 93.67 95.67
formance of the proposed method. In the second step, a con-
Inception 4c 91.63 92.40 91.92 94.87
ditional random field was combined with a multi-cascaded
Inception 4d 96.89 94.09 95.07 92.30
CNN to refine the segmentation result, which was tested
Inception 4e 96.74 95.79 95.56 95.23
on the BraTS 2013, 2015, and 2018 datasets, and produced
Inception 5a 91.02 93.76 92.34 95.76
significant brain tumor segmentation results. This method
Inception 5b 95.97 95.61 95.09 96.98
required 1.5–3 min to segment the whole brain. Using the
Figshare dataset, the authors in Rehman et al. (2019) stud-
ied three convolutional neural networks, including AlexNet,
layers were studied for the extraction of features and the lay- VGGNet, and GoogleNet, for the classification of brain
ers that outperformed the network were explored. Tables 8 tumors as gliomas, meningiomas, and pituitary tumors. For
and 9 report the results. stable and efficient feature learning using MRI images, two
Furthermore, a comparative study was conducted on the types of transfer learning approaches were used, including
performance of the inception layers of the two proposed fine tuning and freezing. The results showed that VGGNet
architectures. It may be observed from the obtained results, with fine tuning achieved the highest classification accu-
shown in Tables 8 and 9, that the performance of the incep- racy, at 98.69%. However, the proposed framework had a
tion layers of the second model was better than that of the high time complexity. The authors of Deepak and Ameer
first model. The second model involves a deeper network (2019) developed a GoogleNet-based architecture for effec-
than the first model, which enhances the performance of the tive feature extraction, and then used deep transfer learning
entire network. to classify brain MRI images containing three categories of
brain tumors, including gliomas, meningiomas, and pituitary
tumors. Their proposed approach used the Figshare MRI
5 Comparison with the states‑of‑the‑art dataset and achieved a mean classification accuracy of 98%.
methods This method was shown to be suitable for classification of
MRI datasets from limited images. The accuracy signifi-
In this section, we compare our proposed method with previ- cantly decreased on larger image datasets. In Toğaçar et al.
ous studies that used the same brain tumor types, and dif- (2020), the authors proposed a new CNN model called Brain
ferent network architectures and parameters. The compara- MRNet for the classification of brain tumors in MRI images,
tive analysis presented in Table 10 shows that our proposed which was based on attention modules and hyper column
network achieved state-of-the-art accuracy and provided the methods. Their proposed attention modules employed data

Table 10  Comparative analysis Models Dataset Best accuracy

of proposed method and
previous related works Hu et al. (2019) BraTS 2013, 2015, and 2018 96.8%
Rehman et al. (2019) Figshare 98.69%
Deepak and Ameer (2019) Figshare 98%
Toğaçar et al. (2020) Free accessible MRI images 96.05%
Kumar and Mankame (2020) BraTS database and SimBraTS 96.3%
Zhang et al. (2020) BraTS 2017, BraTS 2018 and BraTS 2019 91.43%
Sultan et al. (2019) two publicly available datasets 96.13%
Gumaei et al. (2019) dataset comprises of 3064 brain images 94.23%
Huang et al. (2020) publicly available dataset 95.49%
Our Proposed Model FigShare and BraTS 2018 1st CNN on Fig-
Share: 97.3%
2nd CNN on
BraTS 2018:
96.5%

13
An MRI‑based deep learning approach for efficient classification of brain tumors

augmentation approaches to choose an area of interest. The (2020). The used a dataset consisting of 1426 glioma images,
extracted features were retained in the last layer of the CNN 708 meningiomas, and 930 pituitary images. The proposed
model using the hyper column method, which focused on model obtained 95.49% accuracy on the task of classify-
selecting the best and most appropriate features. The authors ing brain tumors into three different categories. Table 10
used a publicly available MRI image dataset that included presents a comparison of the proposed approach with the
155 tumor images and 98 normal samples. On the task of existing related works. Our proposed model achieved prom-
classifying MRI images as tumorous or normal, the Brain- ising classification results, but it is still necessary to use a
MRNet model achieved 96.05% accuracy. In Kumar and larger dataset for testing, with more information and details
Mankame (2020), a new method called Dolphin-SCA was of the patients such as age, race, and health condition, which
presented, which focused on a deep CNN model for efficient could extend the portability of the proposed system to other
classification, and on increasing accuracy compared to of medical applications.
prior methods. Preprocessing was performed on the input Figure 18 shows a graphical illustration of the compara-
MRI image first, followed by segmentation using a dolphin tive analysis of the proposed model with state-of-the-art
sine cosine (Dolphin-SCA) algorithm. Power local binary models.
patterns and mathematical methods were used to derive
effective and insightful features. The extracted features
were then classified as tumorous or normal using a deep 6 Discussion and ablation study
convolutional neural network in the final stage. They con-
ducted experiments using two dataset, including the BraTS In the following section, we examine the impact of various
database, which comprised 65 MRI images and SimBraTS. parameters and essential elements on the efficiency and per-
The proposed scheme achieved a maximum accuracy of formance of the proposed model.
96.3%. An attention gate model was explored for brain tumor
segmentation in Zhang et al. (2020), and a novel attention 6.1 Pre‑processing
gate residual U-Net model (AGResU-Net) was developed
to enhance brain tumor segmentation efficiency. AGResU- Pre-processing is the first stage that has a significant influ-
Net was presented as a single U-Net architecture combin- ence on the classification of brain tumors through the deep
ing attention gates and residual modules to remove salient CNN model. A possible alternative approach in Tustison
features and suppress noisy and vague features. The devel- et al. (2014) was also studied and compared to evaluate the
oped framework was verified using the BraTS 2017, BraTS effect of the pre-processing step on the classification pro-
2018, and BraTS 2019 datasets. The suggested structure cess. The N4ITK procedure was used to perform field-biased
outperformed the baseline models of U-Net and ResU-Net
in the experiments. The authors of Sultan et al. (2019) pre-
sented a deep learning model for the classification of brain
tumors using two publicly available brain MRI datasets. In
total, they used 3064 and 516 brain MRI images in the two
datasets. The former dataset was used to classify tumors as
gliomas, meningiomas, or pituitary tumors, while the lat-
ter was used to classify gliomas as Grade II, Grade III or
Grade IV. The deep learning model achieved an accuracy of
96.13% on the task of classifying tumors as gliomas, men-
ingiomas, or pituitary. The authors of Gumaei et al. (2019)
proposed a hybrid feature extraction approach for extracting
efficient features, and then used a regularized extreme learn-
ing machine to classify the features in brain tumor images.
In the first step, features were extracted and then reduced in
dimension through principal component analysis and finally
classified based on an extreme learning machine. The dataset
includes 3064 brain images of three types of brain tumors,
including gliomas, meningiomas, and pituitary tumors. Their
proposed approach achieved 94.23% classification accuracy.
The authors proposed a CNN model focused on complex
networks with modified activation functions for the clas- Fig. 18  Graphical representation of the comparative analysis of the
sification of brain tumors using MRI images in Huang et al. proposed model with state-of-the-art

13
 E. U. Haq et al.

Table 11  Study of key components of the proposed method

Method Metrics
Tumor type Sensitivity Specificity PPV NPV Accuracy (%)

Proposed 1st CNN Architecture using Figshare dataset Glioma 0.960 0.975 0.967 0.970 96.9
Meningioma 0.965 0.979 0.943 0.987 97.7
Pituitary 0.950 0.982 0.959 0.978 97.2
Proposed 2nd CNN Architecture using BRATS 2018 dataset HGG 0.967 0.945 0.951 0.962 95.6
LGG 0.956 0.978 0.916 0.989 97.4
Combined 0.906 0.988 0.966 0.965 96.5
Using pre-processing as in (Dey et al. 2015) Glioma 0.942 0.954 0.952 0.964 94.70
Meningioma 0.950 0.962 0.912 0.953 95.35
Pituitary 0.936 0.951 0.924 0.935 96.10
HGG 0.955 0.938 0.945 0.950 95.8
LGG 0.903 0.965 0.880 0.957 95.3
Combined 0.837 0.954 0.943 0.941 96.8
Without data augmentation Glioma 0.943 0.950 0.950 0.960 94.2
Meningioma 0.920 0.963 0.918 0.948 95.6
Pituitary 0.935 0.965 0.935 0.940 95.7
HGG 0.951 0.932 0.940 0.947 93.8
LGG 0.923 0.963 0.890 0.950 94.8
Combined 0.878 0.972 0.945 0.944 95.1
Using ReLu as Activation function Glioma 0.955 0.962 0.950 0.963 95.6
Meningioma 0.959 0.958 0.937 0.950 96.3
Pituitary 0.943 0.973 0.945 0.953 96.5
HGG 0.956 0.937 0.942 0.949 94.8
LGG 0.938 0.968 0.908 0.960 96.3
Combined 0.887 0.980 0.960 0.958 95.2
Without CRFs Glioma 0.962 0.968 0.960 0.964 95.7
Meningioma 0.967 0.971 0.934 0.970 96.8
Pituitary 0.954 0.975 0.950 0.968 96.1
HGG 0.970 0.940 0.942 0.956 94.2
LGG 0.959 0.968 0.907 0.969 96.8
Combined 0.910 0.975 0.957 0.954 95.1

normalization in each MRI sequence, resulting in a unit vari- 6.2 Data augmentation

ance and zero mean for each MRI sequence. In the training
process of the CNN with this pre-processing, it was observed Data augmentation is a commonly used tool for CNN appli-
that the initial and final learning rates should be kept rela- cations when available samples in a dataset are very lim-
tively low at roughly 2.9 × 10−4 and 2.9 × 10−6 , respectively, ited in number. However, we encountered a large number of
to avoid any possible divergence in the optimization. The samples for both cancerous and healthy tissues in analyzing
results shown on Table 11, demonstrate that pre-processing MRI images. Most studies in the literature (Zikic 2014) on
using the (Nyul et al. 2000) intensity normalization approach brain tumor classification have avoided relying on data aug-
greatly improved performance on various evaluation met- mentation. The authors in Havaei et al. (2017) explored data
rics. These metrics show greater improvement on identifying augmentation in their proposed model, but found it unpro-
HGG and LGG, indicating that the proposed pre-processing ductive. Different types of data augmentation methods based
improved glioma grade identification and classification, on rotation, flipping, and translation have been proposed to
which is commonly considered a difficult and challenging increase the number of samples and evaluate their effect on
task. Pre-processing plays a key role in the improvements the performance of CNN classification. In rotation-based
demonstrated by proposed method on detection and clas- data augmentation, multiples of 90° were used to increase
sification of brain tumors. The removal of pre-processing the number of samples. For efficient and productive CNN
reduces the performance of the proposed CNN models. training, the input images are mirrored from the horizontal

13
An MRI‑based deep learning approach for efficient classification of brain tumors

and vertical directions in the flip data augmentation process. proposed deep CNN can be reproducible for larger-scale or
We examined the impact of data augmentation on the results multi-center data using one of two methods, either a fusion
of various evaluation metrics. Data augmentation greatly of multimodal imaging sequences at the feature learning
increased the efficiency of the proposed model based on the level or a fusion of features of multimodal imaging sequence
evaluation metrics, as shown in Table 11. within the network. In the first approach, multimodal images
are integrated to learn a combined feature map and then fed
6.3 Activation function into the deep CNN model to support the classification task.
In the second mode, each of multiple multimodal images
When the unit of the ReLU activation function is not active, are given separately as input to the deep CNN model, and
the gradients always yield zero, resulting in slow conver- an individual feature map is obtained for each multimodal
gence. This would lead to a negative training outcome for sequence. The outputs of the last convolutional layers of
the CNN model. We examined the impact of the ReLU func- each multimodal sequence are fused together and given
tion on the proposed model. The authors in Raschke et al. as input to the fully connected layer for efficient tumor
(2019) proposed the LReLU activation function to eliminate classification.
the deficiencies of ReLU. In our proposed work, we simply
replaced ReLU with LReLU. This was verified as effective
based on the results shown in Table 11, which indicate that 7 Conclusion
LReLU simply enhanced the performance of both CNN
models on the FigShare and BraTS 2018 datasets, in terms In this study, we have presented two deep architectures for
of evaluation metrics. extracting more discriminative multi-scale features used in
the detection and classification of various forms of brain
6.4 Conditional random field tumors, i.e., gliomas, meningiomas, and pituitary tumors,
as well as HGG and LGG glioma grades, using Figshare
From Table 11, it may be observed that the CNN model and BraTS 2018 datasets. In particular, different network
with CRFs achieved higher results than only using the CNN architectures have been proposed, and their consequences on
model alone. Only the sensitivity results for the CNN model the detection and classification performance were studied,
with CRFs were slightly lower than the results of the CNN and it was observed that high performance could be obtained
model without CRFs. Therefore, the results show that clas- by a deeper architecture fused with CRFs. In summary, we
sification based on the CNN model achieved higher sensi- proposed GoogLeNet-based architectures for the identifi-
tivity, and post-processing with CRFs efficiently refined the cation and precise classification of various brain tumors.
classification results based on various evaluation metrics. In We deployed two deep configurations of GoogLeNet archi-
summary, the CNN effectively classified brain tumors, and tecture on MRI images of the Figshare dataset to identify
post-processing based on CRFs improved and refined the and classify gliomas, meningiomas, and pituitary tumors
classification results. and classify grades of glioma as HGG or LGG. We began
with the pre-processing stage, including bias field correc-
6.5 Multi‑sequence MRI tion, intensity normalization, and contrast normalization.
In our proposed model, we also used intensity normaliza-
Raschke et al. (2019) performed a retroactive study of multi- tion to address the variability caused by multi-site multi-
modal MRI for patients’ diagnosed with gliomas. Magnetic scanner acquisition of MRI images. Brain tumors show a
resonance spectroscopic imaging (MRSI) is a method used high degree of variability in terms of anatomical structures
to label pure low-grade or HGG tumors across all mMRI and spatial localization, so we have used data augmentation
images. Data from a healthy control group and from patients techniques to deal with such variations. We augmented our
with brain metastasis were further used to define normal training dataset by spinning, flipping, and rotating the MRI
brain and edema, respectively. Three different parameters, slices in Figshare and BraTS 2018 datasets. Data augmenta-
including the intensity-normalized proton density, p and q tion techniques enhanced performance and effectiveness on
diffusion maps, and T2-weighted images were exploited for deep learning-based brain tumor detection and classifica-
each individual tissue in order to extract probability density tion tasks, although these methods have not been thoroughly
distributions (PDD). A whole-brain tissue map was gener- explored in the relevant literature. Finally, compared to the
ated based on superpixel segmentation and Bayesian infer- ReLu activation function, the LReLu activation function was
ence. Their results revealed that the total volume of lesions more effective in the training of the proposed deep model.
was detected automatically from the tissue map and HGG An RPN efficiently selected optimal bounding boxes, which
volume was resolved as grade III or grade IV gliomas. Con- were further used by the proposed model to identify brain
sidering multimodal imaging data, the performance of our tumor regions. Using the testing set, better accuracy was

13
 E. U. Haq et al.

obtained for the detection and classification of different brain Brain Tumor Segmentation Challenge (2014) Ilastik for multi-modal
tumors. The first CNN Architecture used for the Figshare brain tumor segmentation, Proceedings MICCAI BraTS (Brain
Tumor Segmentation Challenge)
dataset to classify brain tumors as gliomas, meningiomas, or Cheng (2017) Brain tumor dataset. figshare. Dataset. https://​doi.​org/​
pituitary tumors achieved an accuracy of 97.3% and yielded 10.​6084/​m9.​figsh​are.​15124​27.​v5
a DSC 95.8%, while the second CNN Architecture achieved Cheng J, Huang W, Cao S, Yang R, Yang W, Yun Z, Wang Z, Feng Q
an accuracy of 96.5% with a DSC 94.3%, using the BraTS (2015) Correction: enhanced performance of brain tumor classi-
cation via tumor region augmentation and partition. PLoS ONE
2018 dataset to classify glioma grades as HGG or LGG. In 10:e0144479. https://​doi.​org/​10.​1371/​journ​al.​pone.​01444​79
addition, the current limitation associated with the proposed Deepak S, Ameer PM (2019) Brain tumor classication using deep CNN
model is that it cannot be reused for classification tasks features via transfer learning. Comput Biol Med 111:103345.
on datasets with smaller images. To avoid this issue, the https://​doi.​org/​10.​1016/j.​compb​iomed.​2019.​103345
Dey N, Ashour AS, Beagum S, Pistola DS, Gospodinov M, Gospodi-
model can be trained on a larger dataset to efficiently handle nova ЕP, Tavares JM (2015) Parameter optimization for local
smaller datasets. The other concern is the decreased classi- polynomial approximation based intersection condence interval
fication performance of the proposed network in the case of filter using genetic algorithm: an application for brain MRI image
receiving erroneous information on incoming patients across de-noising. J Imaging 1:60–84. https://​doi.​org/​10.​3390/​jimag​
ing10​10060
various imaging modalities. This limitation may lead us to Dieleman S, Willett KW, Dambre J (2015) Rotation-invariant convo-
explore an integrated approach comprising iterative segmen- lutional neural networks for galaxy morphology prediction. Mon
tation and classification, which is a promising direction for Not R Astron Soc 450:1441–1459. https://​doi.​org/​10.​1093/​mnras/​
future studies. stv632
Dong et al (2017) Automatic Brain Tumor Detection and Segmentation
Using U-Net Based Fully Convolutional Networks. In: Annual
Acknowledgements This work was supported in part by Shenzhen Sci- Conference on Medical Image Understanding and Analysis.
ence and Technology Project (No. JCYJ20200821152629001) Springer, Cham (2017)
Dvorak P, Menze B (2015) Structured prediction with convolutional
Data availability The data that support the findings of this study are neural networks for multimodal brain tumor segmentation,
openly available in the Figshare dataset https://​doi.​org/​10.​6084/​m9.​ MICCAI-BRATS
figsh​are.​15124​27.​v5and the BRATS 2018 dataset https://​www.​med.​ Ertosun MG, Rubin DL (2015) Automated grading of gliomas using
upenn.​edu/​sbia/​brats​2018/​regis​trati​on.​html. deep learning in digital pathology images: a modular approach
with ensemble of convolutional neural networks. In: AMIA Annu
Declarations Symp Proc
Estienne T, Lerousseau M, Vakalopoulou M, Andres EA, Battistella E,
Carre A, Chandra S, Christodoulidis S, Sahasrabudhe M, Sun R,
Conflict of interest The authors declare that they have no conflicts of Robert C, Talbot H, Paragios N, Deutsch E (2020) Deep learning
interest. based concurrent brain registration and tumor segmentation. Front
Comput Neurosci. https://​doi.​org/​10.​3389/​fncom.​2020.​00017
Goetz M, Weber C, Binczyk F, Polanska J, Tarnawski R, Bobek-Bille-
References wicz B, Koethe U, Kleesiek J, Stieltjes B, Maier-Hein KH (2016)
DALSA: domain adaptation for supervised learning from sparsely
annotated MR images. IEEE Trans Med Imaging 35:184–196.
Abiwinanda N, Hanif M, Hesaputra ST, Handayani A, Mengko TR https://​doi.​org/​10.​1109/​tmi.​2015.​24630​78
(2018) Brain tumor classication using convolutional neural net- Gryska E, Schneiderman J, Björkman-Burtscher I, Heckemann RA
work. In: IFMBE proceedings. Springer Singapore, pp 183–189. (2021) Automatic brain lesion segmentation on standard mag-
https://​doi.​org/​10.​1007/​978-​981-​10-​9035-6_​33 netic resonance images: a scoping review. BMJ Open 11:e042660.
Afshar P, Mohammadi A, Plataniotis KN (2018) Brain tumor type clas- https://​doi.​org/​10.​1136/​bmjop​en-​2020-​042660
sication via capsule networks. In: 2018 25th IEEE international Gumaei A, Hassan MM, Hassan MR, Alelaiwi A, Fortino G (2019) A
conference on image processing (ICIP), IEEE. https://​doi.​org/​10.​ hybrid feature extraction method with regularized extreme learn-
1109/​icip.​2018.​84513​79 ing machine for brain tumor classication. IEEE Access 7:36266–
Agrawal P, Whitaker RT, Elhabian SY (2020) An optimal generative 36273. https://​doi.​org/​10.​1109/​access.​2019.​29041​45
model for estimating multi-label probabilistic maps. IEEE Trans Havaei M, Davy A, Warde-Farley D, Biard A, Courville A, Bengio Y,
Med Imaging 39:2316–2326. https://​doi.​org/​10.​1109/​tmi.​2020.​ Pal C, Jodoin P-M, Larochelle H (2017) Brain tumor segmentation
29689​17 with Deep Neural Networks. Med Image Anal 35:18–31. https://​
Anaraki AK, Ayati M, Kazemi F (2019) Magnetic resonance imaging- doi.​org/​10.​1016/j.​media.​2016.​05.​004
based brain tumor grades classication and grading via convolu- Hu K, Zhang Z, Niu X, Zhang Y, Cao C, Xiao F, Gao X (2018) Reti-
tional neural networks and genetic algorithms. Biocybern Biomed nal vessel segmentation of color fundus images using multiscale
Eng 39:63–74. https://​doi.​org/​10.​1016/j.​bbe.​2018.​10.​004 convolutional neural network with an improved cross-entropy
Bauer S, Wiest R, Nolte L-P, Reyes M (2013) A survey of MRI-based loss function. Neurocomputing 309:179–191. https://​doi.​org/​10.​
medical image analysis for brain tumor studies. Phys Med Biol 1016/j.​neucom.​2018.​05.​011
58:R97–R129 Hu K, Gan Q, Zhang Y, Deng S, Xiao F, Huang W, Cao C, Gao X
Bengio Y, Courville A, Vincent P (2013) Representation learning: (2019) Brain tumor segmentation using multi-cascaded convolu-
a review and new perspectives. IEEE Trans Pattern Anal Mach tional neural networks and conditional random field. IEEE Access
Intell 35:1798–1828. https://​doi.​org/​10.​1109/​tpami.​2013.​50 7:92615–92629. https://​doi.​org/​10.​1109/​access.​2019.​29274​33
Huang Z, Du X, Chen L, Li Y, Liu M, Chou Y, Jin L (2020) Con-
volutional neural network based on complex networks for brain

13
An MRI‑based deep learning approach for efficient classification of brain tumors

tumor image classification with a modified activation function. In: Medical image computing and computer-assisted intervention
IEEE Access 8:89281–89290. https://​doi.​org/​10.​1109/​access.​ MICCAI 2012. Springer Berlin Heidelberg, pp 651–658. https://​
2020.​29936​18 doi.​org/​10.​1007/​978-3-​642-​33418-4_​80
Isin A, Direkoglu C, Sah M (2016) Review of MRI-based Brain tumor Paul JS, Plassard AJ, Landman BA, Fabbri D (2017) Deep learning
image segmentation using deep learning methods. Proc Comput for brain tumor classication. In: Krol A, Gimi B (eds) Medical
Sci 102:317–324. https://​doi.​org/​10.​1016/j.​procs.​2016.​09.​407 imaging 2017: biomedical applications in molecular structural,
Kamnitsas K, Ledig C, Newcombe VFJ, Simpson JP, Kane AD, Menon and functional imaging, SPIE. https://​doi.​org/​10.​1117/​12.​22541​95
DK, Rueckert D, Glocker B (2017) Efficient multi-scale 3D CNN Pavidraa R, Preethi R, Raja NSM, Tamizharasi P, Varthini BP (2020)
with fully connected CRF for accurate brain lesion segmentation. Examination of the brain MRI slices corrupted with induced
Med Image Anal 36:61–78. https://​doi.​org/​10.​1016/j.​media.​2016.​ noise|a study with SGO algorithm. In: Advances in intelligent
10.​004 systems and computing. Springer Singapore, pp 681–690. https://​
Khan MA, Ashraf I, Alhaisoni M, Damasevicius R, Scherer R, Rehman doi.​org/​10.​1007/​978-​981-​15-​5679-1_​66
A, Bukhari SAC (2020) Multimodal brain tumor classication Peng S, Dong Y, Wang W, Hu J, Dong W (2019) The affective facial
using deep learning and robust feature selection: a machine learn- recognition task: the influence of cognitive styles and exposure
ing application for radiologists. Diagnostics 10:565. https://​doi.​ times. J vis Commun Image Represent 65:102674. https://​doi.​org/​
org/​10.​3390/​diagn​ostic​s1008​0565 10.​1016/j.​jvcir.​2019.​102674
Khotanlou H, Colliot O, Atif J, Bloch I (2009) 3D brain tumor seg- Pereira S, Pinto A, Alves V, Silva CA (2016) Brain tumor segmenta-
mentation in MRI using fuzzy classification symmetry analysis tion using convolutional neural networks in MRI images. IEEE
and spatially constrained deformable models. Fuzzy Sets Syst Trans Med Imaging 35:1240–1251. https://​doi.​org/​10.​1109/​tmi.​
160:1457–1473. https://​doi.​org/​10.​1016/j.​fss.​2008.​11.​016 2016.​25384​65
Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classication Prastawa M (2004) A brain tumor segmentation framework based on
with deep convolutional neural networks. Commun ACM 60:84– outlie detection. Med Image Anal 8:275–283
90. https://​doi.​org/​10.​1145/​30653​86 Prastawa M, Bullitt E, Ho S, Gerig G (2003) Robust estimation for
Kumar S, Mankame DP (2020) Optimization driven Deep Convolution brain tumor segmentation. In: Lecture notes in computer science.
Neural Network for brain tumor classification. Biocybern Biomed Springer Berlin Heidelberg, pp 530–537. https://​doi.​org/​10.​1007/​
Eng 40:1190–1204. https://​doi.​org/​10.​1016/j.​bbe.​2020.​05.​009 978-3-​540-​39903-2_​65.
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436– Raja NSM, Fernandes SL, Dey N, Satapathy SC, Rajinikanth V (2018)
444. https://​doi.​org/​10.​1038/​natur​e14539 Contrast enhanced medical MRI evaluation using Tsallis entropy
Li J, Udupa JK, Tong Y, Wang L, Torigian DA (2021) Segmentation and region growing segmentation. J Ambient Intell Humaniz
evaluation with sparse ground truth data: simulating true seg- Comput. https://​doi.​org/​10.​1007/​s12652-​018-​0854-8
mentations as perfect/imperfect as those generated by humans. Rajinikanth V, Satapathy SC, Fernandes SL, Nachiappan S (2017)
Med Image Anal 69:101980. https://​doi.​org/​10.​1016/j.​media.​ Entropy based segmentation of tumor from brain MR images—a
2021.​101980 study with teaching learning based optimization. Pattern Recogn
Lyksborg M, Puonti O, Agn M, Larsen R (2015) An ensemble of 2D Lett 94:87–95. https://​doi.​org/​10.​1016/j.​patrec.​2017.​05.​028
convolutional neural networks for tumor segmentation. In: Image Rajinikanth V, Satapathy SC, Dey N, Vijayarajan R (2018) DWT-
analysis. Springer International Publishing, pp 201–211. https://​ PCA image fusion technique to improve segmentation accuracy
doi.​org/​10.​1007/​978-3-​319-​19665-7_​17. in brain tumor analysis. In: Lecture notes in electrical engineering.
Maas AL, Hannun AY, Ng AY (2013) Rectier nonlinearities improve Springer Singapore, pp 453–462. https://​doi.​org/​10.​1007/​978-​981-​
neural network acoustic models. In: Proceedings of the 30th inter- 10-​7329-8_​46
national conference on machine learning, vol 30. Atlanta, Georgia Raschke F, Barrick TR, Jones TL, Yang G, Ye X, Howe FA (2019)
Menze BH, Jakab A, Bauer S, Kalpathy-Cramer J, Farahani K, Kirby Tissue-type mapping of gliomas. NeuroImage: Clin 21:101648.
J, Burren Y, Porz N, Slotboom J, Wiest R, Lanczi L, Gerstner E, https://​doi.​org/​10.​1016/j.​nicl.​2018.​101648
Weber M-A, Arbel T, Avants BB, Ayache N, Buendia P, Collins Rehman A, Naz S, Razzak MI, Akram F, Imran M (2019) A deep
DL, Cordier N, Corso JJ, Criminisi A, Das T, Delingette H, Demi- learning-based framework for automatic brain tumors classication
ralp C, Durst CR, Dojat M, Doyle S, Festa J, Forbes F, Geremia using transfer learning. Circuits Syst Signal Process 39:757–775.
E, Glocker B, Golland P, Guo X, Hamamci A, Iftekharuddin KM, https://​doi.​org/​10.​1007/​s00034-​019-​01246-3
Jena R, John NM, Konukoglu E, Lashkari D, Mariz JA, Meier R, Satapathy SC, Rajinikanth V (2018) Jaya algorithm guided procedure
Pereira S, Precup D, Price SJ, Raviv TR, Reza SMS, Ryan M, to segment tumor from brain MRI. J Optim 2018:1–12. https://​
Sarikaya D, Schwartz L, Shin H-C, Shotton J, Silva CA, Sousa doi.​org/​10.​1155/​2018/​37380​49
N, Subbanna NK, Szekely G, Taylor TJ, Thomas OM, Tustison Sharma K, Virmani J (2017) A decision support system for classication
NJ, Unal G, Vasseur F, Wintermark M, Ye DH, Zhao L, Zhao B, of normal and medical renal disease using ultrasound images. Int
Zikic D, Prastawa M, Reyes M, Leemput KV (2015) The mul- J Ambient Comput Intell 8:52–69. https://​doi.​org/​10.​4018/​ijaci.​
timodal brain tumor image segmentation benchmark (BRATS). 20170​40104
IEEE Trans Med Imaging 34:1993–2024. https://​doi.​org/​10.​1109/​ Shelhamer E, Long J, Darrell T (2017) Fully convolutional networks
tmi.​2014.​23776​94 for semantic segmentation. IEEE Trans Pattern Anal Mach Intell
MICCAI (2014) Multi-modal brain tumor image segmentation using 39:640–651. https://​doi.​org/​10.​1109/​tpami.​2016.​25726​83
glistr (2014) In: Proceedings of BRATS Challenge–MICCAI Soltaninejad M, Yang G, Lambrou T, Allinson N, Jones TL, Barrick
Narmatha C, Eljack SM, Tuka AARM, Manimurugan S, Mustafa M TR, Howe FA, Ye X (2016) Automated brain tumour detection
(2020) A hybrid fuzzy brain-storm optimization algorithm for and segmentation using superpixel-based extremely randomized
the classification of brain tumor MRI images. J Ambient Intell trees in FLAIR MRI. Int J Comput Assist Radiol Surg 12:183–
Humaniz Comput. https://​doi.​org/​10.​1007/​s12652-​020-​02470-5 203. https://​doi.​org/​10.​1007/​s11548-​016-​1483-3
Nyul LG, Udupa JK, Zhang X (2000) New variants of a method of Soltaninejad M, Yang G, Lambrou T, Allinson N, Jones TL, Barrick
MRI scale standardization. IEEE Trans Med Imaging 19:143–150. TR, Howe FA, Ye X (2018) Supervised learning based multi-
https://​doi.​org/​10.​1109/​42.​836373 modal MRI brain tumour segmentation using texture features
Parisot S, Duau H, Chemouny S, Paragios N (2012) Joint tumor seg- from supervoxels. Comput Methods Programs Biomed 157:69–84.
mentation and dense deformable registration of brain MR images. https://​doi.​org/​10.​1016/j.​cmpb.​2018.​01.​003

13
 E. U. Haq et al.

Subbanna NK, Precup D, Collins DL, Arbel T (2013) Hierarchical Wu Y, Hatipoglu S, Alonso-Alvarez D, Gatehouse P, Li B, Gao Y,
probabilistic gabor and MRF segmentation of brain tumours in Firmin D, Keegan J, Yang G (2021) Fast and automated segmenta-
MRI volumes. In: Advanced information systems engineering. tion for the three-directional multi-slice cine myocardial velocity
Springer Berlin Heidelberg, pp 751–758. https://​doi.​org/​10.​1007/​ mapping. Diagnostics 11:346. https://​doi.​org/​10.​3390/​diagn​ostic​
978-3-​642-​40811-3_​94. s1102​0346
Subbanna N, Precup D, Arbel T (2014) Iterative multilevel MRF lever- Xavier Glorot YB (2010) Understanding the difficulty of training deep
aging context and voxel information for brain tumour segmenta- feedforward neural networks. In: Proceedings of the thirteenth
tion in MRI. In: 2014 IEEE conference on computer vision and international conference on artificial intelligence and statistics,
pattern recognition, IEEE. https://​doi.​org/​10.​1109/​cvpr.​2014.​58 PMLR. 9:249–256
Sultan HH, Salem NM, Al-Atabany W (2019) Multi-classication of Yang Q, Zhang H, Xia J, Zhang X (2021) Evaluation of magnetic
brain tumor images using deep neural network. IEEE Access resonance image segmentation in brain low-grade gliomas
7:69215–69225. https://​doi.​org/​10.​1109/​access.​2019.​29191​22 using support vector machine and convolutional neural network.
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Quant Imaging Med Surg 11:300–316. https://​doi.​org/​10.​21037/​
Vanhoucke V, Rabinovich A (2015) Going deeper with convolu- qims-​20-​783
tions. In: 2015 IEEE conference on computer vision and pattern Zacharaki EI, Wang S, Chawla S, Yoo DS, Wolf R, Melhem ER,
recognition (CVPR), IEEE. https://​doi.​org/​10.​1109/​cvpr.​2015.​ Davatzikos C (2009) Classication of brain tumor type and grade
72985​94 using MRI texture and shape in a machine learning scheme. Magn
Thaha MM, Kumar KPM, Murugan BS, Dhanasekeran S, Vijayakar- Reson Med 62:1609–1618. https://​doi.​org/​10.​1002/​mrm.​22147
thick P, Selvi AS (2019) Brain tumor segmentation using convo- Zhang J, Jiang Z, Dong J, Hou Y, Liu B (2020) Attention gate ResU-
lutional neural networks in MRI images. J Med Syst. https://​doi.​ net for automatic MRI brain tumor segmentation. IEEE Access
org/​10.​1007/​s10916-​019-​1416-0 8:58533–58545. https://​doi.​org/​10.​1109/​access.​2020.​29830​75
Toğaçar M, Ergen B, Cömert C (2020) BrainMRNet: brain tumor Zhang D, Huang G, Zhang Q, Han J, Han J, Yu Y (2021) Cross-
detection using magnetic resonance images with a novel convo- modality deep feature learning for brain tumor segmentation.
lutional neural network model. Med Hypotheses 134:109531. Pattern Recogn 110:107562. https://​doi.​org/​10.​1016/j.​patcog.​
https://​doi.​org/​10.​1016/j.​mehy.​2019.​109531 2020.​107562
Tustison NJ, Shrinidhi KL, Wintermark M, Durst CR, Kandel BM, Gee Zhao X, Wu Y, Song G, Li Z, Zhang Y, Fan Y (2018) A deep learning
JC, Grossman MC, Avants BB (2014) Optimal symmetric multi- model integrating FCNNs and CRFs for brain tumor segmenta-
modal templates and concatenated random forests for supervised tion. Med Image Anal 43:98–111. https://​doi.​org/​10.​1016/j.​media.​
brain tumor segmentation (simplied) with ANTsR. Neuroinfor- 2017.​10.​002
matics 13:209–225. https://​doi.​org/​10.​1007/​s12021-​014-​9245-2 Zhao Y-X, Zhang Y-M, Liu C-L (2020) Bag of tricks for 3D MRI brain
Urban G, Bendszus M, Hamprecht F, Kleesiek J (2014) Multi-modal tumor segmentation. In: Brain-lesion: glioma multiple sclerosis,
brain tumor segmentation using deep convolutional neural net- stroke and traumatic brain injuries. Springer International Publish-
works. Miccai-Bratss. 31–35 ing, pp 210–220. https://​doi.​org/​10.​1007/​978-3-​030-​46640-4_​20.
Wang G, Li W, Zuluaga MA, Pratt R, Patel PA, Aertsen M, Doel T, Zhou X, Li X, Hu K, Zhang Y, Chen Z, Gao X (2021) ERV-Net: an
David AL, Deprest J, Ourselin S, Vercauteren T (2018) Interactive efficient 3D residual neural network for brain tumor segmenta-
medical image segmentation using deep learning with image-spe- tion. Expert Syst Appl 170:114566. https://​doi.​org/​10.​1016/j.​
cific fine tuning. IEEE Trans Med Imaging 37:1562–1573. https://​ eswa.​2021.​114566
doi.​org/​10.​1109/​tmi.​2018.​27917​21 Zikic (2014) Brain tumor segmentation with deep neural networks.
Wang G, Zuluaga MA, Li W, Pratt R, Patel PA, Aertsen M, Doel T, MICCAI Multimodal Brain Tumor Segmentation Challenge
David AL, Deprest J, Ourselin S, Vercauteren T (2019) DeepI- (BraTS), pp 31–35
GeoS: a deep interactive geodesic framework for medical image Zikic D, Ioannou Y, Brown M, Criminisi A (2014) Segmentation of
segmentation. IEEE Trans Pattern Anal Mach Intell 41:1559– brain tumor tissues with convolutional neural networks. In: Con-
1572. https://​doi.​org/​10.​1109/​tpami.​2018.​28406​95 ference: MICCAI workshop on multimodal brain tumor segmenta-
Wu W, Li D, Du J, Gao X, Gu W, Zhao F, Feng X, Yan H (2020) An tion challenge (BRATS), Boston, Massachusetts
intelligent diagnosis method of brain MRI tumor segmentation
using deep convolutional neural Network and SVM algorithm. Publisher's Note Springer Nature remains neutral with regard to
Comput Math Methods Med 2020:1–10. https://​doi.​org/​10.​1155/​ jurisdictional claims in published maps and institutional affiliations.
2020/​67893​06

13

View publication stats