Faisal Mahmood

Colorectal cancer is the fourth leading cause of cancer deaths worldwide and the second leading cause in the United States. The risk of colorectal cancer can be mitigated by the identification and removal of premalignant lesions through optical colonoscopy. Unfortunately, conventional colonoscopy misses more than 20% of the polyps that should be removed, due in part to poor contrast of lesion topography. Imaging tissue topography during a colonoscopy is difficult because of the size constraints… Show more

Colorectal cancer is the fourth leading cause of cancer deaths worldwide and the second leading cause in the United States. The risk of colorectal cancer can be mitigated by the identification and removal of premalignant lesions through optical colonoscopy. Unfortunately, conventional colonoscopy misses more than 20% of the polyps that should be removed, due in part to poor contrast of lesion topography. Imaging tissue topography during a colonoscopy is difficult because of the size constraints of the endoscope and the deforming mucosa. Most existing methods make geometric assumptions or incorporate a priori information, which limits accuracy and sensitivity. In this paper, we present a method that avoids these restrictions, using a joint deep convolutional neural network-conditional random field (CNN-CRF) framework. Estimated depth is used to reconstruct the topography of the surface of the colon from a single image. We train the unary and pairwise potential functions of a CRF in a CNN on synthetic data, generated by developing an endoscope camera model and rendering over 100,000 images of an anatomically-realistic colon. We validate our approach with real endoscopy images from a porcine colon, transferred to a synthetic-like domain, with ground truth from registered computed tomography measurements. The CNN-CRF approach estimates depths with a relative error of 0.152 for synthetic endoscopy images and 0.242 for real endoscopy images. We show that the estimated depth maps can be used for reconstructing the topography of the mucosa from conventional colonoscopy images. This approach can easily be integrated into existing endoscopy systems and provides a foundation for improving computer-aided detection algorithms for detection, segmentation and classification of lesions. Show less

To realize the full potential of deep learning for medical imaging, large annotated datasets are required for training. Such datasets are difficult to acquire due to privacy issues, lack of experts available for annotation, underrepresentation of rare conditions and poor standardization. The lack of annotated data has been addressed in conventional vision applications using synthetic images refined via unsupervised adversarial training to look like real images. However, this approach is… Show more

To realize the full potential of deep learning for medical imaging, large annotated datasets are required for training. Such datasets are difficult to acquire due to privacy issues, lack of experts available for annotation, underrepresentation of rare conditions and poor standardization. The lack of annotated data has been addressed in conventional vision applications using synthetic images refined via unsupervised adversarial training to look like real images. However, this approach is difficult to extend to general medical imaging because of the complex and diverse set of features found in real human tissues. We propose a novel framework that uses a reverse flow, where adversarial training is used to make real medical images more like synthetic images, and clinically-relevant features are preserved via self-regularization. These domain-adapted synthetic-like images can then be accurately interpreted by networks trained on large datasets of synthetic medical images. We implement this approach on the notoriously difficult task of depth-estimation from monocular endoscopy which has a variety of applications in colonoscopy, robotic surgery and invasive endoscopic procedures. We train a depth estimator on a large dataset of synthetic images generated using an accurate forward model of an endoscope and an anatomically-realistic colon. Our analysis demonstrates that the structural similarity of endoscopy depth estimation in a real pig colon predicted from a network trained solely on synthetic data improved by 78.7% by using reverse domain adaptation. Show less

Colorectal cancer is a leading cause of cancer deaths worldwide, but mortality can be mitigated by the detection and removal of premalignant lesions. Unfortunately, conventional 2D optical colonoscopy does not capture topographical information of the surface of the mucosa and thus has a high lesion miss rate. In this short paper, we use a joint deep convolutional neural network-conditional random field (CRF) framework for depth estimation from monocular colonoscopy images. Unlike previous… Show more

Colorectal cancer is a leading cause of cancer deaths worldwide, but mortality can be mitigated by the detection and removal of premalignant lesions. Unfortunately, conventional 2D optical colonoscopy does not capture topographical information of the surface of the mucosa and thus has a high lesion miss rate. In this short paper, we use a joint deep convolutional neural network-conditional random field (CRF) framework for depth estimation from monocular colonoscopy images. Unlike previous approaches, this method does not make any geometric assumptions. The estimated depth is used to reconstruct the topography of the surface of the colon. Using digitally generated synthetic endoscopy data and CT-Phantom data, with corresponding ground truth depths, we train the unary and pairwise potential functions of a conditional random filed in a joint network. Results show that this approach can estimate depths for test data with an 84% accuracy. We show that estimated depth maps can be used for reconstructing the topography of the mucosa from real colonoscopy images. This topographical information can be used for improving learning-based algorithms for detection, segmentation and identification of lesions. Show less

Sparsity exploiting image reconstruction (SER) methods have been extensively used with Total Variation (TV) regularization for tomographic reconstructions. Local TV methods fail to preserve texture details and often create additional artifacts due to over-smoothing. Non-Local TV (NLTV) methods have been proposed as a solution to this but they either lack continuous updates due to computational constraints or limit the locality to a small region. In this paper, we propose Adaptive Graph-based TV… Show more

Sparsity exploiting image reconstruction (SER) methods have been extensively used with Total Variation (TV) regularization for tomographic reconstructions. Local TV methods fail to preserve texture details and often create additional artifacts due to over-smoothing. Non-Local TV (NLTV) methods have been proposed as a solution to this but they either lack continuous updates due to computational constraints or limit the locality to a small region. In this paper, we propose Adaptive Graph-based TV (AGTV). The proposed method goes beyond spatial similarity between different regions of an image being reconstructed by establishing a connection between similar regions in the entire image regardless of spatial distance. As compared to NLTV the proposed method is computationally efficient and involves updating the graph prior during every iteration making the connection between similar regions stronger. Moreover, it promotes sparsity in the wavelet and graph gradient domains. Since TV is a special case of graph TV the proposed method can also be seen as a generalization of SER and TV methods. Show less

Molecular structure determination is important for understanding functionalities and dynamics of macromolecules, such as proteins and nucleic acids. Cryo-electron tomography is a technique that can be used to determine structures of individual macromolecules, thus providing snapshots of their native conformations. Such 3D reconstructions encounter several types of imperfections due to missing, corrupted and lowcontrast data. In this study, we demonstrate that extending the reconstruction space,… Show more

Molecular structure determination is important for understanding functionalities and dynamics of macromolecules, such as proteins and nucleic acids. Cryo-electron tomography is a technique that can be used to determine structures of individual macromolecules, thus providing snapshots of their native conformations. Such 3D reconstructions encounter several types of imperfections due to missing, corrupted and lowcontrast data. In this study, we demonstrate that extending the reconstruction space, which increases the dimensionality of the linear system being solved during reconstruction, facilitates the separation of signal and noise. A considerable amount of the noise associated with collected projection data arises independently from the geometric constraint of image formation, whereas the solution to the reconstruction problem must satisfy such geometric constraints. Increasing the dimensionality thereby allows for a redistribution of such noise within the extended reconstruction space, while the geometrically constrained approximate solution stays in an effectively lower dimensional subspace. Employing various tomographic reconstruction methods with regularization capability we performed extensive simulation and testing and observed that enhanced dimensionality significantly improves the accuracy of the reconstruction. Our results were validated with reconstructions of colloidal silica nanoparticles as well as P. falciparum Erythrocyte Membrane Protein 1 (PfEMP1). Although the proposed method is used in the context of Cryo-ET, the method is general and can be extended to a variety of other tomographic modalities. Show less

Many image processing algorithms are implemented in a combination of spatial and frequency domains. The fast Fourier transform (FFT) is the workhorse of such algorithms. One limitation of the FFT is artefacts that result from the implicit periodicity within the spatial domain. A new periodic plus smooth decomposition has recently been proposed for removing such artefacts, although this comes at the cost of an additional 2D FFT. In this paper, we restructure the decomposition to enable it to be… Show more

Many image processing algorithms are implemented in a combination of spatial and frequency domains. The fast Fourier transform (FFT) is the workhorse of such algorithms. One limitation of the FFT is artefacts that result from the implicit periodicity within the spatial domain. A new periodic plus smooth decomposition has recently been proposed for removing such artefacts, although this comes at the cost of an additional 2D FFT. In this paper, we restructure the decomposition to enable it to be calculated with a single 1D FFT, which can significantly accelerate artefact free Fourier transformation. The cost of this acceleration is a small amount of additional storage to hold the representation of the smooth image component. Show less

Limited data and low dose constraints are common problems in a variety of tomographic reconstruction paradigms which lead to noisy and incomplete data. Over the past few years sinogram denoising has become an essential pre-processing step for low dose Computed Tomographic (CT) reconstructions. We propose a novel sinogram denoising algorithm inspired by the modern field of signal processing on graphs. Graph based methods often perform better than standard filtering operations since they can… Show more

Limited data and low dose constraints are common problems in a variety of tomographic reconstruction paradigms which lead to noisy and incomplete data. Over the past few years sinogram denoising has become an essential pre-processing step for low dose Computed Tomographic (CT) reconstructions. We propose a novel sinogram denoising algorithm inspired by the modern field of signal processing on graphs. Graph based methods often perform better than standard filtering operations since they can exploit the signal structure. This makes the sinogram an ideal candidate for graph based denoising since it generally has a piecewise smooth structure. We test our method with a variety of phantoms and different reconstruction methods. Our numerical study shows that the proposed algorithm improves the performance of analytical filtered back-projection (FBP) and iterative methods ART (Kaczmarz) and SIRT (Cimmino).We observed that graph denoised sinogram always minimizes the error measure and improves the accuracy of the solution as compared to regular reconstructions. Show less

The purpose of the study is to examine the effect of subliminal priming in terms of the perception of images influenced by words with positive, negative, and neutral emotional content, through electroencephalograms (EEGs). Participants were instructed to rate how much they like the stimuli images, on a 7-point Likert scale, after being subliminally exposed to masked lexical prime words that exhibit positive, negative, and neutral connotations with respect to the images. Simultaneously, the EEGs… Show more

The purpose of the study is to examine the effect of subliminal priming in terms of the perception of images influenced by words with positive, negative, and neutral emotional content, through electroencephalograms (EEGs). Participants were instructed to rate how much they like the stimuli images, on a 7-point Likert scale, after being subliminally exposed to masked lexical prime words that exhibit positive, negative, and neutral connotations with respect to the images. Simultaneously, the EEGs were recorded. Statistical tests such as repeated measures ANOVAs and two-tailed paired-samples t-tests were performed to measure significant differences in the likability ratings among the three prime affect types; the results showed a strong shift in the likeness judgment for the images in the positively primed condition compared to the other two. The acquired EEGs were examined to assess the difference in brain activity associated with the three different conditions. The consistent results obtained confirmed the overall priming effect on participants’ explicit ratings. In addition, machine learning algorithms such as support vector machines (SVMs), and AdaBoost classifiers were applied to infer the prime affect type from the ERPs. The highest classification rates of 95.0% and 70.0% obtained respectively for average-trial binary classifier and average-trial multi-class further emphasize that the ERPs encode information about the different kinds of primes.
Show less

Two-Dimensional (2D) Discrete Fourier Transform (DFT) is a basic and computationally intensive algorithm, with a vast variety of applications. 2D images are, in general, non-periodic, but are assumed to be periodic while calculating their DFTs. This leads to cross-shaped artifacts in the frequency domain due to spectral leakage. These artifacts can have critical consequences if the DFTs are being used for further processing. In this paper we present a novel FPGA-based design to calculate… Show more

Two-Dimensional (2D) Discrete Fourier Transform (DFT) is a basic and computationally intensive algorithm, with a vast variety of applications. 2D images are, in general, non-periodic, but are assumed to be periodic while calculating their DFTs. This leads to cross-shaped artifacts in the frequency domain due to spectral leakage. These artifacts can have critical consequences if the DFTs are being used for further processing. In this paper we present a novel FPGA-based design to calculate high-throughput 2D DFTs with simultaneous edge artifact removal. Standard approaches for removing these artifacts using apodization functions or mirroring, either involve removing critical frequencies or a surge in computation by increasing image size. We use a periodic-plus-smooth decomposition based artifact removal algorithm optimized for FPGA implementation, while still achieving real-time (~23 frames per second) performance for a 512x512 size image stream. Our optimization approach leads to a significant decrease in external memory utilization thereby avoiding memory conflicts and simplifies the design. We have tested our design on a PXIe based Xilinx Kintex 7 FPGA system communicating with a host PC which gives us the advantage to further expand the design for industrial applications.
Show less

The research presented in this article investigates the influence of subliminal prime words on peoples' judgment about images, through electroencephalograms (EEGs). In this cross domain priming paradigm, the participants are asked to rate how much they like the stimulus images, on a 7-point Likert scale, after being subliminally exposed to masked lexical prime words, with EEG recorded simultaneously. Statistical analysis tools are used to analyze the effect of priming on behavior, and machine… Show more

The research presented in this article investigates the influence of subliminal prime words on peoples' judgment about images, through electroencephalograms (EEGs). In this cross domain priming paradigm, the participants are asked to rate how much they like the stimulus images, on a 7-point Likert scale, after being subliminally exposed to masked lexical prime words, with EEG recorded simultaneously. Statistical analysis tools are used to analyze the effect of priming on behavior, and machine learning techniques to infer the primes from EEGs. The experiment reveals strong effects of subliminal priming on the participants' explicit rating of images. The subjective judgment affected by the priming makes visible change in event-related potentials (ERPs); results show larger ERP amplitude for the negative primes compared with positive and neutral primes. In addition, Support Vector Machine (SVM) based classifiers are proposed to infer the prime types from the average ERPs, which yields a classification rate of 70%. Show less