Richard J. Chen

Big healthcare AI news yesterday as Tempus AI received the first-ever CPT code for an algorithm-only analysis of previously-sequenced, laboratory developed test transcriptomic data. TLDR: AI molecular phenotyping has a new business model. The PurISTSM test uses RNA profiling to identify molecular subtypes in patients with advanced pancreatic ductal adenocarcinoma, which may inform first-line therapy selection and management. While I'm skeptical that payer balance sheets are going to solve the "who pays for healthcare AI" problem, the reimbursement pipes for clinical AI are maturing. I am surprised to see more progress for usage-based vs. performance-based models given the many challenges with FFS economics. It will be interesting to see how payers assess the "medical necessity" and therefore reimbursability of clinical AI as these models come to market. https://t.co/eLe18SmihV

420

9 Comments

Psssst healthcare folks (and job seekers on the prowl)! AI in healthcare is a hot topic these days—so here’s a quick rundown of the major trends (credit to HLTH USA). ➡️ Imaging & Diagnostics: AI in medical imaging boosts diagnostic speed, accuracy, and efficiency, aiding clinicians in early disease detection and secure workflows. ➡️ Virtual Scribes & Assistance: AI-powered virtual scribes reduce admin tasks, letting providers focus on patients while improving workflow efficiency and documentation. ➡️ Patient Engagement & Care Management: AI enhances patient engagement through personalized care, streamlined communication, and coordinated care management. ➡️ Clinical Decision Support: AI supports data-driven decisions, improving patient care with real-time insights and evidence-based guidance. ➡️ Personalized Care & Workflow Automation: AI personalizes treatment with clinical/genetic insights, optimizing care for complex diseases and improving outcomes. ➡️ Predictive Analytics & RCM: AI-driven analytics enhance healthcare by predicting trends, improving outcomes, and streamlining billing for scalable, value-based care. ➡️ Healthcare Consulting & Administration: AI-powered consulting drives efficiency, innovation, and superior outcomes by transforming healthcare operations.

21

👉🏼 Evaluating ChatGPT-3.5 and ChatGPT-4 in Clinical Decision-Making: A Comparative Analysis 🤓 Adi Lahat https://lnkd.in/eHh7HQWH 🔍 Focus on data insights: - Both GPT models received high mean scores (4.4 for GPT-4 and 4.1 for GPT-3.5). - GPT-4 outperformed GPT-3.5 across all rating dimensions. - Seniors consistently rated responses higher than residents for both models. 💡 Main outcomes and implications: - ChatGPT shows promise in assisting physicians with medical issues. - Potential to enhance diagnostics, treatments, and ethics. - Integration into clinical workflows should complement human expertise. 📚 Field significance: - Artificial Intelligence - Clinical Decision-Making - Health Care - Chatbot Systems - Medical Ethics 🗄️: [#ArtificialIntelligence #ClinicalDecisionMaking #HealthCare #ChatbotSystems #MedicalEthics]

3

👉🏼 ChatGPT's Accuracy in Triage of Nephrology Patient Communications 🤓 Justin H Pham 👇🏻 https://lnkd.in/eYkqgU4h 🔍 Focus on data insights: - 📈 In the first trial, ChatGPT correctly triaged 93% of messages, demonstrating strong reliability. - 🔍 The AI system maintained consistency across trials, with a similar accuracy rate in both assessments. - ⚖️ Internal agreement between ChatGPT's responses was high, indicating robust performance and minimal variability in results. 💡 Main outcomes and implications: - 🤖 High accuracy in triaging suggests that AI could be effectively integrated into outpatient settings to streamline patient communication. - ⏱️ Potential reduction in response times for patient messages, leading to improved patient care and operational efficiency. - 🏥 Encourages further exploration of AI applications in healthcare for enhanced decision-making and resource management. 📚 Field significance: - 🔬 Highlights the role of artificial intelligence in transforming patient care pathways within nephrology and beyond. - 🌐 Sets a precedent for future studies assessing AI capabilities in various medical specialties. - 📊 Supports the development of AI-driven tools aimed at optimizing triage processes and improving clinical outcomes. 🗄️: [#AI #Triage #Nephrology #PatientCare #HealthcareInnovation #ChatGPT #DataInsights #Efficiency]

1

We’re excited to share our latest research at Google Health to expand capabilities of medical generative AI models to include 3D scan reporting, risk prediction from genomics, and much more. 🧠 For the first time, our new Med-Gemini-3D model can 𝗿𝗲𝗮𝗱 𝟯𝗗 𝘀𝗰𝗮𝗻𝘀 like head CTs, answer questions about them, and even write radiology reports. 🩻 Our Med-Gemini-2D model demonstrates improved performance in writing chest X-ray radiology reports, exceeding previous state-of-the-art results by up to 12%. 🔬 We explore capabilities across 2D medical images, including X-rays, histopathology slides, and dermatology photos. 🔮 Med-Gemini-Polygenic is our first language model capable of predicting health outcomes based on genomic data. Importantly, our models are trained using free text reports that are freely available for most medical imaging, almost eliminating the need for costly expert labelling. We hope this will make it easier, faster, cheaper to train high-quality medical AI models in the future than has been possible in the past. We believe this research is a promising step forward in the field of medical AI - the ability to interpret complex multimodal medical data is a crucial step toward enabling high quality personalised care at scale. Of course, these are early results, and further research is required before safe clinical use - if you’re interested in joining us on this journey, please get in touch by filling in the form here: https://shorturl.at/aejpE Full paper at https://lnkd.in/eAb8F-Bk - we hope you enjoy it. Congratulations to our amazing team from Google Research and Google DeepMind: Lin Yang, Shawn Xu, Andrew Sellergren, Timo Kohlberger, Ira Ktena, PhD, Kendall Park, Ryutaro Tanno, David Barrett, Wei-Hung Weng, Atilla K., Akshay Goel, M.D., Arnav Agharwal, Nick George, Cory McLean, Yuchen Zhou, Farhad Hormozdiari, Eric Wang, David Steiner, Faruk Ahmed, Tiam Jaroensri, Ellery Wulczyn, Fayaz Jamil, Theo Guidroz, Yang Wang, Siyuan Qiao, Yun Liu, S. Sara Mahdavi, Charles Lau, MD, MBA, Dr Sreenivasa Raju Kalidindi, Mozziyar Etemadi, Jorge Cuadros OD PhD, Greg Sorensen, Shruthi Prabhakara, Daniel Tse, Shravya Shetty, Greg Corrado, Katherine Chou, Yossi Matias, S. M. Ali Eslami, David Fleet, Joëlle Barral, Daniel Golden, Shekoofeh Azizi, Rory Pilgrim, Christopher Kelly

1,982

84 Comments

AI Proteins, a biotechnology company that uses computational de novo protein design to create therapeutic miniproteins, announced last week an agreement with Bristol Myers Squibb (BMS) to discover and develop novel miniprotein-based therapeutics using AI Proteins’ discovery platform. The deal could be worth up to $400M plus royalties on net sales. Miniproteins comprise: a peptide that folds into a highly stable structure, optional disulfide bonds, canonical amino acids without post translational modifications, entirely novel composition of matter designed to be non-immunogenic, and more. Notably, the company says, “because drug-like properties are designed into the miniproteins from the start, every molecule with optimized activity is ready to begin pre-clinical studies.” AI in drug discovery is one of the most active areas of biotech deal-making right now. There has been a steady stream of deals the last two years, but just within the last few months:

2

👉🏼 Chat-ePRO: Development and pilot study of an electronic patient-reported outcomes system based on ChatGPT 🤓 Zikang Chen 👇🏻 https://lnkd.in/ejE_73_2 🔍 Focus on data insights: - Chat-ePRO correctly generates conversation based on PRO forms with a success rate of 95.7%. - The system accurately extracts PRO data instantaneously from conversations with a Macro-F1 score of 0.95. - Subjective evaluations from doctors (>70%) indicate that Chat-ePRO comprehends questions and consistently generates responses. 💡 Main outcomes and implications: - Chat-ePRO demonstrates a higher response rate (90%) and longer interaction time (10.86 s/turn) compared to other methods in the pilot study. - Feasibility of utilizing prompt engineering algorithms to drive Large Language Models (LLMs) for ePRO data collection tasks is validated. - The Chat-ePRO system effectively enhances patient compliance. 📚 Field significance: - Development of Chat-ePRO showcases the potential of LLMs in improving user compliance in electronic Patient-Reported Outcome (ePRO) systems. - The study highlights the importance of leveraging advanced algorithms like prompt engineering for enhancing data collection processes in healthcare settings. 🗄️: [#Chat-ePRO, #patient-reported outcomes, #data insights, #healthcare technology]

2

👉🏼 Educational Limitations of ChatGPT in Neurosurgery Board Preparation 🤓 Andrew Y Powers 👇🏻 https://lnkd.in/e3GyFwdr 🔍 Focus on data insights: - ChatGPT accuracy was 50.4% across various subcategories. - Performance varied, with the highest accuracy in pharmacology (81.2%) and the lowest in vascular (32.9%). - ChatGPT performed worse than humans overall and in several specific subcategories. 💡 Main outcomes and implications: - ChatGPT's accuracy was below the passing threshold for the ABNS primary exam. - The study highlights limitations in ChatGPT's ability to effectively teach neurosurgical board material. - Concordance and insight scores were positively associated with accuracy. 📚 Field significance: - This study provides a comprehensive assessment of ChatGPT's accuracy and explanatory quality in answering neurosurgical board exam questions. - The findings underscore the challenges in using AI tools like ChatGPT for educational purposes in neurosurgery. 🗄️: [#neurosurgery #ChatGPT #educationallimitations #datainsights]

1

1 Comment

As the discourse shifts from models to compound AI systems / agents, we need better AI benchmarks to evaluate multi-modal and multi-step task performance, especially in healthcare and life sciences. When we wrote the first paper demonstrating ChatGPT's performance on the USMLE, we chose the US Medical Licensing Exam as a benchmark for accessibility, speed, and ease. This benchmark was never intended to represent AI model performance on real-world clinical tasks. Today, I still see so many research teams and startups using benchmarks (like the USMLE) that are ill-fitted for assessing the true clinical or scientific performance and utility of the models they are developing for real-world contexts. Benchmark development may be seen as a "less sexy" area of research, but it is of paramount importance. Years after the rise of the transformer, we still lack adequate benchmarks for so many single-step tasks in biomedicine. With compound AI systems (i.e., architectures that integrate multiple AI models to perform complex tasks) emerging, we need new benchmarks for agentic behaviors. I'd argue that developing an agent with novel capabilities without at least proposing a companion benchmark (if an industry standard does not yet exist) may hinder the adoption of said agent, especially for high-stakes workflows. Designing more benchmarks that capture/simulate real-world clinical and scientific workflows will help us mitigate the major discrepancies observed between in silico and in vivo performance and better support safe + effective deployment of AI in biomedicine. There are already brilliant people focused here, and we need more. DMs are open if you're researching or working in this area of multi-step/multi-modal benchmarking in healthcare and life sciences! #healthcare #ai #artificialintelligence #generativeai

100

18 Comments

Qualitative metrics from the biomedical literature for evaluating large language models in clinical decision-making: a narrative review - Most frequently used LLMs include GPT-3.5, GPT-4, Bard, LLaMa/Alpaca-based models, and Bing Chat. - Key evaluation criteria for LLM outputs are accuracy, completeness, appropriateness, insight, and consistency. - There is significant variation in how studies report findings and assess LLM performance; standardized reporting metrics are needed. DOI: https://lnkd.in/e3iikhzJ DOI: doi:10.1186/s12911-024-02757-z

1

I’m excited to kick off The State of AI in Drug Discovery event with a Keynote Fireside Chat and 5,000+ attendees at 8:10am PT / 11:10am ET alongside Fay Lin, PhD. Together, we’ll explore how AI is transforming drug discovery end-to-end—from mapping biology to designing new chemical entities—redefining the future of pharma. I’m honored to serve alongside David Baker, who will share insights from his groundbreaking work using computers to design novel proteins. The advancements from his team at the University of Washington, Institute for Protein Design and the achievements of Demis Hassabis and John Jumper at Google DeepMind—now recognized with The Nobel Prize — have profoundly accelerated the field of protein design. At Recursion, we are leveraging multi-omic datasets—spanning phenomics, genomics, transcriptomics, and increasingly proteomics, and beyond— to create AI-powered Maps of Biology. These maps allow us to uncover novel biological insights and identify new biology and therapeutic targets that were previously out of reach. By combining this biological knowledge with cutting-edge AI-powered molecular design, we are crafting molecules that have the potential to become transformative medicines. In development, we leverage AI-powered solutions to advance predictive trial design, streamline execution, and operationalize workflows. AI helps us make smarter, faster decisions—from optimizing patient recruitment to accelerating evidence generation—improving the probability of success at every stage. Discovery and development remain profoundly humbling endeavors, reminding us of the complexity of biology. But through AI, we are unlocking new possibilities to deliver better medicines faster, ensuring treatments reach patients efficiently, with the highest levels of safety, precision, and impact. The secret to our success lies in our people and culture, who are bilingual in both the art of science and the language of AI. This unique blend allows us to navigate the complexities of biology and technology with creativity, precision, and agility. I look forward to sharing more about these innovations and continuing the conversation at the event! Details on the Oct. 30 The State of AI in Drug Discovery event here: https://lnkd.in/ec5nHGQN Read the full article in GEN here: https://lnkd.in/egEjQvtM #ai #techbio #proteins #chemistry #drugdiscovery #data #biology #maps

380

15 Comments