RT Journal Article
SR 00
ID 10.3390/s24185865
A1 Kaul, Chaitanya
A1 Mitchell, Kevin J.
A1 Kassem, Khaled
A1 Tragakis, Athanasios
A1 Kapitany, Valentin
A1 Starshynov, Ilya
A1 Villa, Federica
A1 Murray-Smith, Roderick
A1 Faccio, Daniele
T1 AI-enabled sensor fusion of time-of-flight imaging and mmWave for concealed metal detection
JF Sensors
YR 2024
FD 2024-09
VO 24
IS 18
K1 mmWave radar sensing, multi-modal sensing, information fusion, sensor fusion, mmWave,  deep learning, metal detection.
AB In the field of detection and ranging, multiple complementary sensing modalities may be used to enrich information obtained from a dynamic scene. One application of this sensor fusion is in public security and surveillance, where efficacy and privacy protection measures must be continually evaluated. We present a novel deployment of sensor fusion for the discrete detection of concealed metal objects on persons whilst preserving their privacy. This is achieved by coupling off-the-shelf mmWave radar and depth camera technology with a novel neural network architecture that processes radar signals using convolutional Long Short-Term Memory (LSTM) blocks and depth signals using convolutional operations. The combined latent features are then magnified using deep feature magnification to reveal cross-modality dependencies in the data. We further propose a decoder, based on the feature extraction and embedding block, to learn an efficient upsampling of the latent space to locate the concealed object in the spatial domain through radar feature guidance. We demonstrate the ability to detect the presence and infer the 3D location of concealed metal objects. We achieve accuracies of up to 95% using a technique that is robust to multiple persons. This work provides a demonstration of the potential for cost-effective and portable sensor fusion with strong opportunities for further development.
NO D.F. acknowledges funding from the Royal Academy of Engineering Chairs in Emerging  Technologies program and the UK Engineering and Physical Sciences Research Council (grant no. EP/T00097X/1). R.M.S. and C.K. received funding from EPSRC projects Quantic EP/T00097X/1 and QUEST EP/T021020/1 and from the DIFAI ERC Advanced Grant proposal 101097708, funded by the UK Horizon guarantee scheme as EPSRC project EP/Y029178/1. This work was in part supported by a research gift from Google.
PB MDPI
SN 1424-8220
LK https://eprints.gla.ac.uk/334639/