11 IV April 2023

https://doi.org/10.22214/ijraset.2023.50366
 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.538
Volume 11 Issue IV Apr 2023- Available at www.ijraset.com

Fusion of CT and MRI Medical Images Using Dual

Tree Complex Wavelet Transform
Gajjala Jahnavi1, Alla Deepthi Chowdary2, Ananthula Jagadeesh3, Associate Prof. Kachapuram Basava Raju4
1, 2, 3, 4
Department of Artificial Intelligence, Anurag Group of Institutions, Hyderabad, Telangana, India

Abstract: Combining various medical pictures will improve illness diagnosis accuracy and illustrate the complex link between
them for medical study. Existing approaches take a long time and require a large number of data to train the models. In this
model, we will use multi-stage fusion networks to extract sophisticated information from medical pictures such as Magnetic
Resonance Imaging (MRI) and Computed Tomography (CT). In the suggested approach, we will extract the difficult and
correlated information from each picture using the Dual Tree Complex Wavelet Transform (DTCWT), and then segment the
fused image to obtain the segmented image.
The proposed approach entails, the fusion of multimodal medical pictures may be accomplished using the Dual Tree Complex
Wavelet Transform, which converts the original medical image to grayscale and decomposes it before extracting the wavelet
coefficients using DTCWT. Following that, wavelet approximation is utilized to produce the fused coefficients. To obtain a final
fused picture, the Inverse Dual Tree Complicated Wavelet Transform is performed. Additionally, segmentation is carried out in
order to provide a segmented image for better visual representation. The quality of the final fused picture can be increased using
the proposed strategy.
Keywords: Image Fusion, Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Dual Tree Complex Wavelet
Transform (DTCWT), Positron Emission Tomography (PET), Inverse Dual Tree Complex Wavelet Transform (DTCWT).

I. INTRODUCTION
Medical images like X-rays, Positron Emission Tomography (PET), Single Photon Emission Computed Tomography (SPECT),
Magnetic Resonance Imaging (MRI), and Computed Tomography (CT) will not provide as much detail for medical research and
diagnosis as detailed information. They might provide a significant amount of detailed clinical data. Medical imaging systems will
provide different medical information regarding tissues that are complicated in most cases. For example, X-rays are used for
identify bone injuries and bone fractures, CT images will give the elaborated information of internal organs, tumors and blood
vessels.
MRI is used to provide information about tissues. Whereas the SPECT will show how blood flows to tissues and organs, and PET
helps to reveal how the tissues and organs are functioning. As the expansion of clinical usage of diverse medical imaging systems,
the merging of multi- modality pictures plays a significant role in the medical imaging sector. Various medical imaging modalities
can produce scans with both complementary and redundant information. The integration of medical pictures can provide clinical
information that was not visible in the individual scans.
The project’s main goal is to create an efficient fusion approach that uses the Dual Tree Complex Wavelet Transform to fuse single
or multi-modal pictures. In a single image, all important information from several source photographs might be gathered. The
Discrete Wavelet Transform (DWT) is the most often used approach for picture fusion. The resulting image will comprise spectral
and directional information; furthermore, the orientation data will include more precise information on horizontal, vertical, and
diagonal directions.
Some features of the discrete wavelet transform are perfect reconstruction, no redundancy, very low computation but it shows
severe shift dependence and directionality. Because to tiny movements in the source picture, the shift variance will cause errors in
fused images, and due to poor polarity, this original picture would be hard to evaluate geometric elements such as contours and
edges.
To tackle this, the suggested technique would employ Dual Tree Complex Wavelet Transform to merge various medical pictures
using DTCWT. DTCWT will address the drawbacks of DWT. The DTCWT will give greater directionality and shift variance,
making it easier to analyze the source image’s edges and contours. Its higher shift variance and directionality attributes result in a
fully functional picture fusion tool.

©IJRASET: All Rights are Reserved | SJ Impact Factor 7.538 | ISRA Journal Impact Factor 7.894 | 1424
 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.538
Volume 11 Issue IV Apr 2023- Available at www.ijraset.com

II. RELATED WORK

Table 1. Similar Works

Authors Journal Approach Advantages
Publication

X. Xu, Y. Wang, S. Chen Biomedical Fused medical images using the This method has
Signal Processing and Control discrete fractional wavelet potential for use in a
transform. range of medical
applications, such as
disease diagnosis and
treatment planning.

Diwakar, M., Tripathi, A., Joshi, Materials Today: Proceedings Used stationary wavelet Effective for medical
K., Sharma, A., Singh, transform (SWT) and discrete image fusion,
P., Memoria, M., & Kumar wavelet transform (DWT). depending on the
specific application and
the desired fusion
quality.

Z. Chao, X. Duan, S. Jia, X. Applied Soft Computing Used the discrete stationary The method has
Guo, H. Liu, and F. Jia, wavelet transform (SWT) and an potential for use in a
enhanced radial basis function range of medical
(RBF) neural network. imaging applications.

Richa, K. Kaur, and P. Singh International Journal of Online Used the discrete wavelet Evaluates quality
and Biomedical Engineering transform (DWT) and Principal metrics such as peak
Component Analysis (PCA). signal-to-noise
ratio (PSNR) and
structural similarity
index (SSIM),

S. Karamzadeh and M. Cumhuriyet Science Journal Used the discrete wavelet Effectiveness of fusion
Moshantat transform approach. scheme based on
wavelet transform.

III. EXISTING METHOD

Figure 1: Fundamental phases for a basic pixel level Image Fusion strategy [7].

©IJRASET: All Rights are Reserved | SJ Impact Factor 7.538 | ISRA Journal Impact Factor 7.894 | 1425
 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.538
Volume 11 Issue IV Apr 2023- Available at www.ijraset.com

Figure 1 shows the key phases of the general pixel-level image fusion approaches. They are as follows: 1) A mathematical
transform is performed to the input images to obtain the transform coefficients 2) The fusion process is employed to create the fused
coefficients, and 3) an inverse transform is performed to the fused coefficients to create the final fused image.
Imaging techniques such as Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) in the field of medical imaging
have supplied doctors with information on the structural characteristics, soft tissue, and so on of the human body. Distinct imaging
technologies retain different properties, and different sensors receive different imaging information from the same component.
Traditional medical image fusion methods are classified into two categories: spatial domain and transform domain. Discrete
Wavelet Transform was used to merge multimodal medical pictures such as computed tomography (CT) and magnetic resonance
imaging (MRI). The advantages of transparent bone are combined by combining MRI and CT. To compensate for the absence of
data in a single imaging, information in CT pictures and clear soft tissue in MRI images are used. A guided filtering-based MRI and
CT fusion technique (GF) is suggested. The fused picture not just preserves the edge information of the source image but also
extracts feature information, resolving the edge degree and clarity issues. The visual inspection of fusion findings shows that
brightness and correlation have improved noticeably. The discrete wavelet transform may generate diverse input frequency signals
while keeping stable output appropriate location in the temporal and frequency domains, which aids in the preservation of picture
particular information. The wavelet transform overcomes the restrictions of principal component analysis and provides an effective
visual and numerical fusion effect. The source picture is improved and preprocessed, and the intensity component of the CT image
is extracted using the IHS transform, which maintains more biological information and decreases color distortion. To get high- and
low-frequency subbands, the DWT transform is applied to the intensity components of MRI and CT. The high and low- frequency
subbands are fused using distinct fusion criteria, and the fused picture is obtained using the inverse DWT transform. Figure shows a
block schematic of a 1-step 2-D DWT. To fuse the decomposing high-frequency coefficients, the absolute high-value approach is
utilized, the weighted average method is employed, the predator-optimizer is used to calculate and optimize the weights, and lastly,
the inverted transform is used to generate the fused pictures. Fusion is accomplished by the application of two fusion rules. Because
high and low frequency coefficients have various meanings, they were fused using different rules. The first set of principles is that
bigger wavelet coefficients indicate important picture characteristics such as corners and edges, hence larger wavelet coefficients
are preferred. The wavelet factor is the most often used method for fusing features since larger values indicate stronger edges and
are considered as a significant component of relevant information. Low wavelet coefficient values represent source image
approximation, hence averaging is employed to obtain information about both source pictures. An Inverse Discrete Wavelet
Transform is used to get the fused picture after approximating the derived wavelet coefficients. The CT image of a patient brain
diseased by Sarcoma is shown in figure 2 and the MRI image of the same patient is shown in figure 3. These two multimodal
images were fused by applying the Discrete Wavelet Transform algorithm. After the extraction of the wavelet coefficients and
further fusion rule is applied to fuse the coefficients.
After fusing the extracted wavelet coefficients using the derived fusion rules, the Inverse Discrete Wavelet Transform is applied to
obtain the final fused image. The final fused image is shown in figure 4. The problems related with DWT are, it does not provide
sufficient directional information and results in an image with shift variance and additive noise. It also does not preserve time and
frequency information and hence it is not as efficient to represent the enhanced visual representation of the fused image.

Figure 2: Source CT image of a patient diseased by a sarcoma

©IJRASET: All Rights are Reserved | SJ Impact Factor 7.538 | ISRA Journal Impact Factor 7.894 | 1426
 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.538
Volume 11 Issue IV Apr 2023- Available at www.ijraset.com

Figure 3: Source MRI image of a patient diseased by a sarcoma

Figure 4: DWT Fused Image

IV. PROPOSED METHOD

Machine learning applied to radiological imaging is a growing study area that is predicted to expand in coming years. Machine
learning breakthroughs have the ability to recognize and classify complicated patterns from medical images. Machine learning-
based systems have demonstrated similar results to human decision-making in a variety of situations. Machine learning
applications are essential components of future clinical decision-making and monitoring systems. The basic concept beside
machine learning approaches and their implementations in several radiological imaging areas, including medical image
segmentation, brain function studies, and neurological disease diagnosis, as well as computer-aided devices, image restoration, and
content-based image retrieval systems, are covered in this review. We hope that by providing insight into how to use machine
learning-powered applications, clinicians will be able to better prevent and detect diseases.
A great picture fusion technique should incorporate complementary information from the source photos while ignoring undesired
and unexpected aspects. Image fusion may be performed at three different levels: pixel, characteristic, and decision. Following that
is the features level image fusion, also known as the intermediate level image fusion. This technique may represent and evaluate
multi-sensor data for classification. This approach is an important theoretical tool for image classification tasks. The ultimate level
of picture fusion is the decision level, which is the act of merging information at a higher level as well as combining the findings
of several algorithms to obtain a final fused decision. Individual photos are processed for information extraction in this step. MRI,
or Magnetic Resonance Imaging, offers information on the tissue anatomy of the brain but does not provide functional information.
Because the density of protons is high in the neurological system, fat, surrounding tissues, and articular cartilage lesions, the 17
picture is extremely clear and free of artifacts. This has high precision and minimal radiation harm to a human body, and the
benefit of vast amounts of information makes it a valuable tool in clinical diagnostics.

©IJRASET: All Rights are Reserved | SJ Impact Factor 7.538 | ISRA Journal Impact Factor 7.894 | 1427
 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.538
Volume 11 Issue IV Apr 2023- Available at www.ijraset.com

Because the density of protons in bone is quite low, the MRI bone picture is blurry. Individual photos are processed for
information extraction in this step. MRI, or Magnetic Resonance Imaging, provides information on the tissue anatomy of the brain
but does not provide functional information. So because the density of protons is high in the neurological system, fat, surrounding
tissues, and articular cartilage lesions, the 17 image is particularly clear and free of artifacts. This has high precision and negligible
radiation harm to a human body, and the benefit of abundant information makes it a helpful resource in clinical diagnostics.
Because the density of protons in bone is quite low, the MRI bone picture is blurry. The CT picture is referred to as Computed
Tomography imaging. Because bone tissue absorbed more than soft tissue, the bone tissue in the CT picture is extremely apparent.
CT scans reveal less cartilage information, which provides anatomical data. Single-Photon Emission Computed Tomography
(SPECT) is a functional picture that shows the metabolism of tissues or organs as well as the blood flow of arteries and veins. It
gives both benign and malignant tumor information and is widely used in the detection of different tumor illnesses. However,
SPECT has a limited resolution and weak positioning ability. The Anaconda tool is used in combination with Tensorflow, Keras,
Numpy, Seaborn, Opencv, Pandas, Matplotlib, Pytorch, and Python3.The Tensorflow and Keras frameworks are critical for image
fusion. The performance of this image fusion framework utilizing DTCWT is measured by entropy, standard error, peak signal - to
- noise, root mean square error, and fusion factor.

Figure 5: DTCWT Image Fusion Framework [11]

The suggested framework implements Dual Tree Complex Wavelet Transform in picture fusion. The dual tree complex wavelet
transform will address the drawbacks of the Discrete Wavelet Transform (DWT). The DTCWT will give greater directionality and
shift variance, making it easier to analyze the source image’s edges and contours. The DTCWT’s higher shift variance and increased
directionality characteristics result in a fully functional picture fusion tool. Medical imaging systems are beneficial in detecting
numerous abnormalities and disorders in our bodies. Medical imaging systems include magnetic resonance imaging (MRI),
computed tomography (CT), X-rays, 18 Positron Emission Tomography (PET), and single photon emission computed tomography
(SPECT). Each will give distinct details, but none of these technologies will be able to deliver all of the necessary information in a
single image. The solution to this challenge is medical image fusion. A single image might include all of the important information
from several source photos.
Figure 4.1 depicts the steps taken to fuse two source medical pictures using the Dual Tree Complex Wavelet Transform and
generate the fused image. First, register the CT and MRI pictures, then do wavelet decomposition, fuse the two images, then restore
the decomposed images. The basic idea behind wavelet image fusion is to combine the wavelet decompositions of the two original
pictures utilizing fusion algorithms applied to approximations and detailed coefficients. This gives resources for the Wavelet
toolbox analysis program is a collection of several functions built on the technical computer environment PYTHON. It also aids in
the synthesis of image deterministic and random signals using wavelets and wavelet packets in the PYTHON programming
language.

©IJRASET: All Rights are Reserved | SJ Impact Factor 7.538 | ISRA Journal Impact Factor 7.894 | 1428
 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.538
Volume 11 Issue IV Apr 2023- Available at www.ijraset.com

V. MODEL ARCHITECTURE

Figure 6: Proposed Model Architecture

1) Image Dataset
The CT and MRI images are in 512*512 in size, The submitted image is in RGB format. This RGB image is transformed to a
grayscale image with intensity information. The source grayscale images CT and MRI images are then deconstructed using
DTCWT.

Figure 7: image fusion

2) Extracting Features using DTCWT

Following decomposition, the high and low amplitude features will be extracted as wavelet coefficients. Feature extraction is being
used to reduce data dimension and, if the data is large enough, to uncover quiet and complementary characteristics. Because the
medical photos are bigger, PCA is employed to minimize the data here. The PCA will eliminate the unneeded and redundant
information in DTCWT. There are various phases to the PCA analysis. The very first phase is standardization, which aims to
standardize the range of the continuous initial variables. So, each will contribute equally to the analysis. Standardization is
conducted mathematically by comparing the distance between each value and the mean and then dividing by the standard deviation.
The next step is to calculate the covariance matrix; certain variables are highly linked with others, resulting in duplicate information.
As a result, calculate the correlation matrix to analyze the connection. To identify the major components, compute the eigenvectors
and eigenvalues of the correlation matrix. The featured vector is then extracted from all of the calculated principal vectors. Finally,
just use principal to reconstruct the data axes.

3) Image Fusion
The extracted coefficients are approximate. Finally, the approximated wavelet equations are obtained. The estimated coefficients are
now merged to obtain the fused coefficients. To reconstruct the fused picture, we now employ the Inverse dual tree complex wavelet
transform. Figure depicts the combined picture.

©IJRASET: All Rights are Reserved | SJ Impact Factor 7.538 | ISRA Journal Impact Factor 7.894 | 1429
 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.538
Volume 11 Issue IV Apr 2023- Available at www.ijraset.com

4) Image Segmentation
Filtration is used to differentiate between the minutiae of lines, curves, and borders. As a result, the segmentation is done on the
final fused picture. Figure 8 depicts the segmentation results.

Figure 8: image segmentation

VI. CONCLUSION
This paper highlights medical image fusion based on the dual tree wavelet packet transformation in this research. The fused multi
model picture will be computed using the dual tree wavelet transform, which has good specificity of directionality and lower shift
variance. In this study, we suggest using DTCWT to dissect source pictures such as computed tomography (CT) and magnetic
resonance imaging (MRI). Furthermore, coefficient extraction and estimation of extracted coefficients are conducted. The
inversion dual tree wavelet transform is then employed to create the fused image, followed by segmentation to obtain the divided
binary images. As a result, the merged picture of our suggested approach shows more precise tumor borders, tissues, spectral,
contour, and spatial information.

Figure 9: CT input image

Figure 10: MRI input image

Figure 11: Final fused output image

©IJRASET: All Rights are Reserved | SJ Impact Factor 7.538 | ISRA Journal Impact Factor 7.894 | 1430
 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.538
Volume 11 Issue IV Apr 2023- Available at www.ijraset.com

VII. FUTURE ENHANCEMENTS

Our suggested approach of medical image processing is based on the Dual Tree Complex Wavelet Transform (DTCWT), which
has demonstrated that it gives the fused picture with more precise representation of the tumor’s spectrum, geographical, and soft
tissue features. Concerning performance, our suggested technique has high entropy, fusion factor, and maximum signal noise ratio
values. As a result, our suggested algorithm outperforms previous fusion approaches. In the future, block level fusion may be
employed, and the outcomes, such as performance assessment as well as the final fused image in terms of increased visual
representation, can be improved.

REFERENCES
[1] X. Xu, Y. Wang, and S. Chen, “Medical image fusion using discrete fractional wavelet transform,” Biomedical Signal Processing and Control, vol. 27, pp.
103–111, 2016.
[2] Diwakar, M., Tripathi, A., Joshi, K., Sharma, A., Singh, P., Memoria, M., & Kumar, N., “A comparative review: Medical image fusion using SWT and
DWT.”, Materials Today: Proceedings, vol. 37, pp. 3411–3416, 2021.
[3] Z. Chao, X. Duan, S. Jia, X. Guo, H. Liu, and F. Jia, “Medical image fusion via discrete stationary wavelet transform and an enhanced radial basis function
neural network,” Applied Soft Computing, vol. 118, no. 108542, 2022.
[4] Richa, K. Kaur, and P. Singh, “A novel MRI and CT image fusion based on Discrete Wavelet Transform and Principal Component analysis for enhanced
clinical diagnosis”, International Journal of Online and Biomedical Engineering (IJOE), vol. 18, no. 10, pp. 64–82, 2022.
[5] S. Karamzadeh and M. Moshantat, “CT and MRI medical image fusion using discrete wavelet transform,” Cumhuriyet Science Journal, vol. 40, no. 4, pp.
939–945, 2019.
[6] Sruthy S, Dr. Latha Parameswaran, Ajeesh P Sasi, “Image Fusion Technique using DT CWT”, IEEE, pp. 160164, 2018.
[7] S. Li, X. Kang, L. Fang, J. Hu and H. Yin, "Pixel-level image fusion: A survey of the state of the art", Information Fusion, vol. 33, pp. 100-112, 2017.
[8] Karishma C Bhataria, Bhumika K Shah, “A Review of Image Fusion Techniques,” Second International Conference on Computing Methodologies and
Communication (ICCMC), 2018.
[9] Rajasekhar K., Krishna S., Srinivasu V., Neeraj B., Jyothi K., “Image Fusion with CT and MRI Images for improving the quality of Diagnostics”,
International Journal of Engineering Applied Sciences and Technology, vol. 6, no. 12 ,pp.318-322, 2022.
[10] Nayera Nahvi, Deep Mittal, “Medical Image Fusion Using Discrete Wavelet Transform”, International Journal of Engineering Research and Applications, vol.
4, pp.165-170, 2014.
[11] Neha Sharma, “Hybrid Image Fusion implemented in DTCWT”, International Journal of Engineering Technology and Computer Research (IJETCR), vol. 2,
pp. 16-21, 2014.

©IJRASET: All Rights are Reserved | SJ Impact Factor 7.538 | ISRA Journal Impact Factor 7.894 | 1431