Anuj Shah Ph. D.
Santa Clara, California, United States
2K followers
500+ connections
About
Heavily contributed to the Netflix recommendations system for the past 12+ years.
Activity
-
Hi folks, If youre an AI / ML Startup based out of #India or looking for folks there, my cousin Bhavya Chopra , is looking for an #internship in the…
Hi folks, If youre an AI / ML Startup based out of #India or looking for folks there, my cousin Bhavya Chopra , is looking for an #internship in the…
Liked by Anuj Shah Ph. D.
-
We're now accepting applications for Spring or Summer 2025 Machine Learning research internships at Netflix. In our personalization, recommendations,…
We're now accepting applications for Spring or Summer 2025 Machine Learning research internships at Netflix. In our personalization, recommendations,…
Liked by Anuj Shah Ph. D.
-
Come join us tomorrow for an exciting set of talks from AIMon, Zilliz and Airbyte!
Come join us tomorrow for an exciting set of talks from AIMon, Zilliz and Airbyte!
Liked by Anuj Shah Ph. D.
Experience
Education
Publications
-
Spectral archives: extending spectral libraries to analyze both identified and unidentified spectra
Nature Methods
Tandem mass spectrometry (MS/MS) experiments yield multiple, nearly identical spectra of the same peptide in various laboratories, but proteomics researchers typically do not leverage the unidentified spectra produced in other labs to decode spectra they generate. We propose a spectral archives approach that clusters MS/MS datasets, representing similar spectra by a single consensus spectrum. Spectral archives extend spectral libraries by analyzing both identified and unidentified spectra in…
Tandem mass spectrometry (MS/MS) experiments yield multiple, nearly identical spectra of the same peptide in various laboratories, but proteomics researchers typically do not leverage the unidentified spectra produced in other labs to decode spectra they generate. We propose a spectral archives approach that clusters MS/MS datasets, representing similar spectra by a single consensus spectrum. Spectral archives extend spectral libraries by analyzing both identified and unidentified spectra in the same way and maintaining information about peptide spectra that are common across species and conditions. Thus archives offer both traditional library spectrum similarity-based search capabilities along with new ways to analyze the data. By developing a clustering tool, MS-Cluster, we generated a spectral archive from ~1.18 billion spectra that greatly exceeds the size of existing spectral repositories. We advocate that publicly available data should be organized into spectral archives rather than be analyzed as disparate datasets, as is mostly the case today.
-
An efficient data format for mass spectrometry-based proteomics
Journal of American Society for Mass Spectrometry
The diverse range of mass spectrometry (MS) instrumentation along with corresponding proprietary and nonproprietary data formats has generated a proteomics community driven call for a standardized format to facilitate management, processing, storing, visualization, and exchange of both experimental and processed data. To date, significant efforts have been extended towards standardizing XML-based formats for mass spectrometry data representation, despite the recognized inefficiencies associated…
The diverse range of mass spectrometry (MS) instrumentation along with corresponding proprietary and nonproprietary data formats has generated a proteomics community driven call for a standardized format to facilitate management, processing, storing, visualization, and exchange of both experimental and processed data. To date, significant efforts have been extended towards standardizing XML-based formats for mass spectrometry data representation, despite the recognized inefficiencies associated with storing large numeric datasets in XML. The proteomics community has periodically entertained alternate strategies for data exchange, e.g., using a common application programming interface or a database-derived format. However, these efforts have yet to gain significant attention, mostly because they have not demonstrated significant performance benefits over existing standards, but also due to issues such as extensibility to multidimensional separation systems, robustness of operation, and incomplete or mismatched vocabulary. Here, we describe a format based on standard database principles that offers multiple benefits over existing formats in terms of storage size, ease of processing, data retrieval times, and extensibility to accommodate multidimensional separation systems.
-
Machine learning based prediction for peptide drift times in ion mobility spectrometry
Bioinformatics
Motivation: Ion mobility spectrometry (IMS) has gained significant traction over the past few years for rapid, high-resolution separations of analytes based upon gas-phase ion structure, with significant potential impacts in the field of proteomic analysis. IMS coupled with mass spectrometry (MS) affords multiple improvements over traditional proteomics techniques, such as in the elucidation of secondary structure information, identification of post-translational modifications, as well as…
Motivation: Ion mobility spectrometry (IMS) has gained significant traction over the past few years for rapid, high-resolution separations of analytes based upon gas-phase ion structure, with significant potential impacts in the field of proteomic analysis. IMS coupled with mass spectrometry (MS) affords multiple improvements over traditional proteomics techniques, such as in the elucidation of secondary structure information, identification of post-translational modifications, as well as higher identification rates with reduced experiment times. The high throughput nature of this technique benefits from accurate calculation of cross sections, mobilities and associated drift times of peptides, thereby enhancing downstream data analysis. Here, we present a model that uses physicochemical properties of peptides to accurately predict a peptide's drift time directly from its amino acid sequence. This model is used in conjunction with two mathematical techniques, a partial least squares regression and a support vector regression setting.
Results: When tested on an experimentally created high confidence database of 8675 peptide sequences with measured drift times, both techniques statistically significantly outperform the intrinsic size parameters-based calculations, the currently held practice in the field, on all charge states (+2, +3 and +4). -
Applications in Data-Intensive Computing
Advances in Computers: Elsevier
The total quantity of digital information in the world is growing at an alarming rate. Scientists and engineers are contributing heavily to this data “tsunami” by gathering data using computing and instrumentation at incredible rates. As data volumes and complexity grow, it is increasingly arduous to extract valuable information from the data and derive knowledge from that data. Addressing these demands of ever-growing data volumes and complexity requires game-changing advances in software…
The total quantity of digital information in the world is growing at an alarming rate. Scientists and engineers are contributing heavily to this data “tsunami” by gathering data using computing and instrumentation at incredible rates. As data volumes and complexity grow, it is increasingly arduous to extract valuable information from the data and derive knowledge from that data. Addressing these demands of ever-growing data volumes and complexity requires game-changing advances in software, hardware, and algorithms. Solution technologies also must scale to handle the increased data collection and processing rates and simultaneously accelerate timely and effective analysis results. This need for ever faster data processing and manipulation as well as algorithms that scale to high-volume data sets have given birth to a new paradigm or discipline known as “data-intensive computing.” In this chapter, we define data-intensive computing, identify the challenges of massive data, outline solutions for hardware, software, and analytics, and discuss a number of applications in the areas of biology, cyber security, and atmospheric research.
-
An Architecture for Real Time Data Acquisition and Online Signal Processing for High Throughput Tandem Mass Spectrometry
IEEE
Independent, greedy collection of data events using simple heuristics results in massive over-sampling of the prominent data features in large-scale studies over what should be achievable through ¿intelligent", online acquisition of such data. As a result, data generated are more aptly described as a collection of a large number of small experiments rather than a true large-scale experiment. Nevertheless, achieving ¿intelligent¿, online control requires tight interplay between…
Independent, greedy collection of data events using simple heuristics results in massive over-sampling of the prominent data features in large-scale studies over what should be achievable through ¿intelligent", online acquisition of such data. As a result, data generated are more aptly described as a collection of a large number of small experiments rather than a true large-scale experiment. Nevertheless, achieving ¿intelligent¿, online control requires tight interplay between state-of-the-art, data-intensive computing infrastructure developments and analytical algorithms. In this paper, we propose a Software Architecture for Mass spectrometry-based Proteomics coupled with Liquid chromatography Experiments (SAMPLE) to develop an ¿intelligent¿ online control and analysis system to significantly enhance the information content from each sensor (in this case, a mass spectrometer). Using online analysis of data events as they are collected and decision theory to optimize the collection of events during an experiment, we aim to maximize the information content generated during an experiment by the use of pre-existing knowledge to optimize the dynamic collection of events.
-
An Extensible, Scalable Architecture for Managing Bioinformatics Data and Analyses
IEEE
Systems biology research demands the availability of tools and technologies that span a comprehensive range of computational capabilities, including data management, transfer, processing, integration, and interpretation. To address these needs, we have created the Bioinformatics Resource Manager (BRM), a scalable, flexible, and easy to use tool for biologists to undertake complex analyses. This paper describes the underlying software architecture of the BRM that integrates multiple commodity…
Systems biology research demands the availability of tools and technologies that span a comprehensive range of computational capabilities, including data management, transfer, processing, integration, and interpretation. To address these needs, we have created the Bioinformatics Resource Manager (BRM), a scalable, flexible, and easy to use tool for biologists to undertake complex analyses. This paper describes the underlying software architecture of the BRM that integrates multiple commodity platforms to provide a highly extensible and scalable software infrastructure for bioinformatics. The architecture integrates a J2EE 3-tier application with an archival Experimental Data Management System, the GAGGLE framework for desktop tool integration, and the MeDICi Integration Framework for high-throughput data analysis workflows. This architecture facilitates a systems biology software solution that enables the entire spectrum of scientific activities, from experimental data access to high throughput processing and analysis of data for biologists and experimental scientists.
-
Brief Communication: A feature vector integration approach for a generalized support vector machine pairwise homology algorithm
Journal Computational Biology and Chemistry
Due to the exponential growth of sequenced genomes, the need to quickly provide accurate annotation for existing and new sequences is paramount to facilitate biological research. Current sequence comparison approaches fail to detect homologous relationships when sequence similarity is low. Support vector machine (SVM) algorithms approach this problem by transforming all proteins into a feature space of equal dimension based on protein properties, such as sequence similarity scores against a…
Due to the exponential growth of sequenced genomes, the need to quickly provide accurate annotation for existing and new sequences is paramount to facilitate biological research. Current sequence comparison approaches fail to detect homologous relationships when sequence similarity is low. Support vector machine (SVM) algorithms approach this problem by transforming all proteins into a feature space of equal dimension based on protein properties, such as sequence similarity scores against a basis set of proteins or motifs. This multivariate representation of the protein space is then used to build a classifier specific to a pre-defined protein family. However, this approach is not well suited to large-scale annotation. We have developed a SVM approach that formulates remote homology as a single classifier that answers the pairwise comparison problem by integrating the two feature vectors for a pair of sequences into a single vector representation that can be used to build a classifier that separates sequence pairs into homologs and non-homologs. This pairwise SVM approach significantly improves the task of remote homology detection on the benchmark dataset, quantified as the area under the receiver operating characteristic curve; 0.97 versus 0.73 and 0.70 for PSI-BLAST and Basic Local Alignment Search Tool (BLAST), respectively.
-
A support vector machine model for the prediction of proteotypic peptides for accurate mass and time proteomics
Bioinformatics
Motivation: The standard approach to identifying peptides based on accurate mass and elution time (AMT) compares profiles obtained from a high resolution mass spectrometer to a database of peptides previously identified from tandem mass spectrometry (MS/MS) studies. It would be advantageous, with respect to both accuracy and cost, to only search for those peptides that are detectable by MS (proteotypic).
Results: We present a support vector machine (SVM) model that uses a simple…Motivation: The standard approach to identifying peptides based on accurate mass and elution time (AMT) compares profiles obtained from a high resolution mass spectrometer to a database of peptides previously identified from tandem mass spectrometry (MS/MS) studies. It would be advantageous, with respect to both accuracy and cost, to only search for those peptides that are detectable by MS (proteotypic).
Results: We present a support vector machine (SVM) model that uses a simple descriptor space based on 35 properties of amino acid content, charge, hydrophilicity and polarity for the quantitative prediction of proteotypic peptides. Using three independently derived AMT databases (Shewanella oneidensis, Salmonella typhimurium, Yersinia pestis) for training and validation within and across species, the SVM resulted in an average accuracy measure of 0.8 with a SD of <0.025. Furthermore, we demonstrate that these results are achievable with a small set of 12 variables and can achieve high proteome coverage. -
SVM-HUSTLE—an iterative semi-supervised machine learning approach for pairwise protein remote homology detection
Bioinformatics
Motivation: As the amount of biological sequence data continues to grow exponentially we face the increasing challenge of assigning function to this enormous molecular ‘parts list’. The most popular approaches to this challenge make use of the simplifying assumption that similar functional molecules, or proteins, sometimes have similar composition, or sequence. However, these algorithms often fail to identify remote homologs (proteins with similar function but dissimilar sequence) which often…
Motivation: As the amount of biological sequence data continues to grow exponentially we face the increasing challenge of assigning function to this enormous molecular ‘parts list’. The most popular approaches to this challenge make use of the simplifying assumption that similar functional molecules, or proteins, sometimes have similar composition, or sequence. However, these algorithms often fail to identify remote homologs (proteins with similar function but dissimilar sequence) which often are a significant fraction of the total homolog collection for a given sequence. We introduce a Support Vector Machine (SVM)-based tool to detect homology using semi-supervised iterative learning (SVM-HUSTLE) that identifies significantly more remote homologs than current state-of-the-art sequence or cluster-based methods. As opposed to building profiles or position specific scoring matrices, SVM-HUSTLE builds an SVM classifier for a query sequence by training on a collection of representative high-confidence training sets, recruits additional sequences and assigns a statistical measure of homology between a pair of sequences. SVM-HUSTLE combines principles of semi-supervised learning theory with statistical sampling to create many concurrent classifiers to iteratively detect and refine, on-the-fly, patterns indicating homology.
Results: When compared against existing methods for identifying protein homologs (BLAST, PSI-BLAST, COMPASS, PROF_SIM, RANKPROP and their variants) on two different benchmark datasets SVM-HUSTLE significantly outperforms each of the above methods using the most stringent ROC1 statistic with P-values less than 1e-20. SVM-HUSTLE also yields results comparable to HHSearch but at a substantially reduced computational cost since we do not require the construction of HMMs. -
High-throughput computation of pairwise sequence similarities for multiple genome comparison using ScalaBLAST.
Institute of Electrical and Electronics Engineers, Pisctataway, NJ.
Genome sequence comparisons of exponentially growing data sets form the foundation for the comparative analysis tools provided by community biological data resources such as the integrated microbial genome (IMG) system at the joint genome institute (JGI). For a genome sequencing center to provide multiple-genome comparison capabilities, it must keep pace with exponentially growing collection of sequence data, both from its own genomes, and from public genomes. We present an example of how…
Genome sequence comparisons of exponentially growing data sets form the foundation for the comparative analysis tools provided by community biological data resources such as the integrated microbial genome (IMG) system at the joint genome institute (JGI). For a genome sequencing center to provide multiple-genome comparison capabilities, it must keep pace with exponentially growing collection of sequence data, both from its own genomes, and from public genomes. We present an example of how ScalaBLAST, a high-throughput sequence analysis program, harnesses increasingly critical high-performance computing to perform sequence analysis, enabling, for example, all vs. all BLAST runs across 2 million protein sequences within a day using thousands of processors as opposed to conventional comparison methods that would take years to complete.
-
Brief communication: Integrating subcellular location for improving machine learning models of remote homology detection in eukaryotic organisms
Journal Computational Biology and Chemistry
A significant challenge in homology detection is to identify sequences that share a common evolutionary ancestor, despite significant primary sequence divergence. Remote homologs will often have less than 30% sequence identity, yet still retain common structural and functional properties. We demonstrate a novel method for identifying remote homologs using a support vector machine (SVM) classifier trained by fusing sequence similarity scores and subcellular location prediction. SVMs have been…
A significant challenge in homology detection is to identify sequences that share a common evolutionary ancestor, despite significant primary sequence divergence. Remote homologs will often have less than 30% sequence identity, yet still retain common structural and functional properties. We demonstrate a novel method for identifying remote homologs using a support vector machine (SVM) classifier trained by fusing sequence similarity scores and subcellular location prediction. SVMs have been shown to perform well in a variety of applications where binary classification of data is the goal. At the same time, data fusion methods have been shown to be highly effective in enhancing discriminative power of data. Combining these two approaches in the application SVM-SimLoc resulted in identification of significantly more remote homologs (p-value<0.006) than using either sequence similarity or subcellular location independently.
-
Enabling high-throughput data management for systems biology: The Bioinformatics Resource Manager
Bioinformatics
Summary: The Bioinformatics Resource Manager (BRM) is a software environment that provides the user with data management, retrieval and integration capabilities. Designed in collaboration with biologists, BRM simplifies mundane analysis tasks of merging microarray and proteomic data across platforms, facilitates integration of users’ data with functional annotation and interaction data from public sources and provides connectivity to visual analytic tools through reformatting of the data for…
Summary: The Bioinformatics Resource Manager (BRM) is a software environment that provides the user with data management, retrieval and integration capabilities. Designed in collaboration with biologists, BRM simplifies mundane analysis tasks of merging microarray and proteomic data across platforms, facilitates integration of users’ data with functional annotation and interaction data from public sources and provides connectivity to visual analytic tools through reformatting of the data for easy import or dynamic launching capability. BRM is developed using Java™ and other open-source technologies for free distribution.
-
SEBINI: Software Environment for BIological Network Inference
Bioinformatics
Summary: The Software Environment for BIological Network Inference (SEBINI) has been created to provide an interactive environment for the deployment and evaluation of algorithms used to reconstruct the structure of biological regulatory and interaction networks. SEBINI can be used to compare and train network inference methods on artificial networks and simulated gene expression perturbation data. It also allows the analysis within the same framework of experimental high-throughput expression…
Summary: The Software Environment for BIological Network Inference (SEBINI) has been created to provide an interactive environment for the deployment and evaluation of algorithms used to reconstruct the structure of biological regulatory and interaction networks. SEBINI can be used to compare and train network inference methods on artificial networks and simulated gene expression perturbation data. It also allows the analysis within the same framework of experimental high-throughput expression data using the suite of (trained) inference methods; hence SEBINI should be useful to software developers wishing to evaluate, compare, refine or combine inference techniques, and to bioinformaticians analyzing experimental data. SEBINI provides a platform that aids in more accurate reconstruction of biological networks, with less effort, in less time.
-
Bioinformatic Insights from Metagenomics through Visualization
Proceeding CSB '05 Proceedings of the 2005 IEEE Computational Systems Bioinformatics Conference
Cutting-edge biological and bioinformatics research seeks a systems perspective through the analysis of multiple types of high-throughput and other experimental data for the same sample. Systems-level analysis requires the integration and fusion of such data, typically through advanced statistics and mathematics. Visualization is a complementary computational approach that supports integration and analysis of complex data or its derivatives. We present a bioinformatics visualization prototype…
Cutting-edge biological and bioinformatics research seeks a systems perspective through the analysis of multiple types of high-throughput and other experimental data for the same sample. Systems-level analysis requires the integration and fusion of such data, typically through advanced statistics and mathematics. Visualization is a complementary computational approach that supports integration and analysis of complex data or its derivatives. We present a bioinformatics visualization prototype, Juxter, which depicts categorical information derived from or assigned to these diverse data for the purpose of comparing patterns across categorizations. The visualization allows users to easily discern correlated and anomalous patterns in the data. These patterns, which might not be detected automatically by algorithms, may reveal valuable information leading to insight and discovery. We describe the visualization and interaction capabilities and demonstrate its utility in a new field, metagenomics, which combines molecular biology and genetics to identify and characterize genetic material from multi-species microbial samples.
More activity by Anuj
-
Imagine India 2024 - Empowering Businesses & Inspiring Innovation with Agentic Process Automation! Two incredible days of learning, innovation, and…
Imagine India 2024 - Empowering Businesses & Inspiring Innovation with Agentic Process Automation! Two incredible days of learning, innovation, and…
Liked by Anuj Shah Ph. D.
-
Happy Thanksgiving 🍁 Thankful for good health, warm food, kind people, great learning, and an amazing family 🙏🏻
Happy Thanksgiving 🍁 Thankful for good health, warm food, kind people, great learning, and an amazing family 🙏🏻
Liked by Anuj Shah Ph. D.
-
Excited to share our latest research shows that we can generate speech from a multimodal decoder-only architecture, unifying speech, text and vision…
Excited to share our latest research shows that we can generate speech from a multimodal decoder-only architecture, unifying speech, text and vision…
Liked by Anuj Shah Ph. D.
-
✨ Try our LlamaIndex recipe to improve quality of your LLM apps!
✨ Try our LlamaIndex recipe to improve quality of your LLM apps!
Liked by Anuj Shah Ph. D.
-
💡Getting Enterprise AI into Production Starts with Strong AI Governance Just wrapped up an incredible four-days in Europe, where I had energizing…
💡Getting Enterprise AI into Production Starts with Strong AI Governance Just wrapped up an incredible four-days in Europe, where I had energizing…
Liked by Anuj Shah Ph. D.
-
At Imagine India 2024, we showcased our AI Agent Studio platform, enabling rapid automation development for Indian enterprises. In harnessing the…
At Imagine India 2024, we showcased our AI Agent Studio platform, enabling rapid automation development for Indian enterprises. In harnessing the…
Liked by Anuj Shah Ph. D.
-
https://lnkd.in/gBRV5_Xj Congratulations to Ophir Ronen and the team at CalmWave in securing financing to achieve their mission to create quieter…
https://lnkd.in/gBRV5_Xj Congratulations to Ophir Ronen and the team at CalmWave in securing financing to achieve their mission to create quieter…
Liked by Anuj Shah Ph. D.
-
🏆 Congratulations to all our customer & partner award winners at Imagine India. Thank you for pushing the boundaries of what's possible with #AI &…
🏆 Congratulations to all our customer & partner award winners at Imagine India. Thank you for pushing the boundaries of what's possible with #AI &…
Liked by Anuj Shah Ph. D.
-
🚀 Moumita Bhattacharya, Senior Research Scientist Netflix, explores the evolution of recommender and search ranking systems in large-scale…
🚀 Moumita Bhattacharya, Senior Research Scientist Netflix, explores the evolution of recommender and search ranking systems in large-scale…
Liked by Anuj Shah Ph. D.
-
Reinforcement Learning for Large Scale Recommendation System Very excited to deliver a talk on how to use reinforcement learning for optimizing…
Reinforcement Learning for Large Scale Recommendation System Very excited to deliver a talk on how to use reinforcement learning for optimizing…
Liked by Anuj Shah Ph. D.
-
Join us for this hands on session that will go over how to improve your LLM RAG apps!
Join us for this hands on session that will go over how to improve your LLM RAG apps!
Liked by Anuj Shah Ph. D.
Other similar profiles
Explore collaborative articles
We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.
Explore More