Rahul Bhotika

👉🏼 The Digital Technology in Clinical Medicine: From Calculators to ChatGPT 🤓 Somarajan Anandan 👇🏻 https://lnkd.in/gtRmmq8z 🔍 Focus on data insights: - Utilization of digital technology in clinical medicine - Impact of digital information technology on healthcare delivery 💡 Main outcomes and implications: - Advancements in clinical medicine due to digital technology - Need for balance between technology, clinical expertise, and humanistic values 📚 Field significance: - Regulatory push for improvements through clinical trials - Projects like "World Health Organizations Guidelines for Digital Interventions" and "Food and Drug Administration's Digital Health Center of Excellence" highlighting efforts in digital health services 🗄️: [#digitaltechnology #clinicalmedicine #healthcaredelivery]

1

👉🏼 Evaluation of a Large Language Model on the American Academy of Pediatrics' PREP Emergency Medicine Question Bank 🤓 Sriram Ramgopal 👇🏻 https://lnkd.in/ezHKQ6UX 🔍 Focus on data insights: - 📊 ChatGPT achieved a correct answer rate of 74.5% across 215 questions, indicating effectiveness in PEM knowledge assessment. - 🔄 The model demonstrated consistent performance across three years, suggesting reliability in its responses over time. - 📚 Topic-specific performance revealed strengths in research and medical specialties, while highlighting weaknesses in procedures and toxicology. 💡 Main outcomes and implications: - 🏆 ChatGPT exceeded the minimum passing threshold of 65%, supporting potential integration into medical education curricula. - 💬 Detailed explanations provided by the model can enhance understanding among learners, fostering a deeper grasp of pediatric emergency medicine concepts. - ⚠️ Identified limitations in specific topics may guide future enhancements of LLMs for more comprehensive educational tools. 📚 Field significance: - 🧠 The application of LLMs like ChatGPT could revolutionize medical education, offering dynamic learning resources tailored to various specialties. - 🌍 The findings suggest broader implications for using AI in clinical settings, potentially improving diagnostic capabilities and patient care. - 🔍 Continuous evaluation of LLM performance is essential to ensure quality and adaptivity in rapidly evolving medical landscapes. 🗄️: [#LargeLanguageModels #PediatricEmergencyMedicine #AIinHealthcare #MedicalEducation #DataInsights]

1

I'm incredibly honored to represent Kaiser Permanente as a keynote speaker for the upcoming AI + Health Conference - Stanford University's largest convening for AI in health and medicine - organized by the Stanford Center for Artificial Intelligence in Medicine and Imaging (AIMI). In my remarks, I plan to discuss the critical importance of trust and how we build trust, and why, without it, even the most advanced AI technologies will fall short of transforming health care. I'll definitely be staying for the entire program with dedicated tracks covering "Clinical AI Applications in Use Today" "Integrating AI into Healthcare Settings" "Fundamentals of AI in Healthcare" and "Pioneering Innovations in Health AI." This is an online conference on Dec 10 + 11th. There is a registration fee to join and CME will be provided for physicians. Hope you can join the conference! https://lnkd.in/gHmDfCHS

357

15 Comments

🏥 🔬 KPIs (Key Performance Indicators) for Generative AI Application Output & Performance Evaluation in Healthcare and Life Sciences A successful GenAI adoption strategy would warrant tracking KPIs for all stages of the application design - LLM Evaluation, Prompt Engineering & RAG Architecture, and GenAI Application Output Quality. In this 3-part series, I am covering KPIs for LLM evaluation (https://shorturl.at/fjGTZ), Prompt Engineering & RAG Design (https://shorturl.at/sDIR2), and GenAI Application Output quality. In the healthcare and life sciences industry, leveraging generative AI (GenAI) applications can significantly enhance patient outcomes and research initiatives. To ensure your GenAI applications deliver value, consider tracking and evaluating some of these Key Performance Indicators (KPIs): 1. 📈 User Engagement: Monitor end user interactions with your application, assessing active users, session duration, and retention rates. High engagement signals a valuable, user-friendly application. 2. ⏱ Response Time: Digital transformation and the widespread adoption of social media have raised user expectations from all software applications. Measure the time it takes for your application to respond to user queries, striving for faster response times to improve adoption. 3. 🎯 Successful Completion Rate: Track the number of queries successfully completed by your application, ensuring your application delivers accurate, reliable information to support healthcare and life sciences professionals in their work. Add thumbs up/down buttons to solicit feedback per inference. 4. 🛑 Error Rate: Minimize errors to maintain trust and reliability. Regularly log and monitor failed queries to ensure optimal performance. 5. 🔍 Personalization: Measure the application's ability to adapt its responses to individual users' needs and preferences. Tone and output format style flexibility that is tailored to individual's needs, will lead to better stickiness and adoption. 6. 📊 Scalability: Evaluate your application's capacity to handle increasing demand, enabling seamless adoption across the user base. Well architected review is a must! 7. 📝 Regulatory Compliance: Ensure your GenAI solution adheres to industry regulations and GxP guidelines. Monitoring compliance KPIs is essential for product safety, efficacy, and overall quality. 8. 🤝 Human-in-the-Loop Testing: Incorporating human expertise during testing & validation phases will help refine application performance and build confidence for real-world healthcare and research use. 9. 🔗 Workflow integration: Incorporating the application within existing clinical workflows will encourage purpose-built scope, higher adoption, and opportunities to enhance/evolve the application rapidly. 💬 Share your experiences and insights on tracking performance of GenAI systems in the comments below! 📢 Subscribe to my newsletter for insights on GenAI adoption: https://lnkd.in/g3bdneR7 #genai #ai

14

Interesting paper on Medical Coding using LLMs (gpt3.5) on ~100 diagnosis. Would love to see this rerun on GPT-4o

5

Great to see KP on the list of health systems in the US "that are pioneers in demonstrated, outcomes-based AI solutions." While so many companies are tripping over themselves in positioning themselves as "AI-first" the secret for KP's success is that we're decidedly NOT AI-first. We're a people-first organization. Our core mission is delivering high-quality, affordable and equitable care for our 12.5M members and to make KP the best place to work for our 230k+ employees. We deploy technologies to drive those outcomes and we use AI, sparingly and cautiously, when simpler technologies can't get the job done. AI is also a team-sport and KP's success in this area reflects the work of thousands of employees across the organization. People like Jennie Shin who led the national IT team deploying our ambient scribe technology to 25,000 clinicians or Anna Davis and Carol Cain who led our quality assurance testing for our roll-out. And it's particularly gratifying to be recognized alongside my close colleague and friend Vincent Liu. His leadership in AI at KP has been both sustained and truly outcomes-driven. There are many other physician leaders that also deserve to be recognized - Khang Nguyen, Kristine Lee, MD, Brian Hoberman MD MBA, Dr. Ainsley MacLean, Chris Cable, MD, Dinh Nguyen, MD, MSHI and so many more.

177

15 Comments

We’re excited to share Optum Rx is the first to develop a new capability that helps address a major provider pain point: prior authorizations. Prior authorization is an important checkpoint helping patients receive coverage for safe, effective care supported by the most up-to-date clinical guidelines. But the process can add manual, time-consuming tasks to providers’ already heavy workloads. Together with our partners, Surescripts and Cleveland Clinic, we developed Optum Rx PreCheck Prior Authorization, an innovative solution that helps automate the prior authorization process for select drugs, including GLP-1s – from 8.5 hours to under 30 seconds. Learn more about Optum’s commitment to providing a better provider and patient experience: http://spr.ly/6045S4vph

5

Today the most important article in chatgpt and "A text-based conversational agent for asthma support: Mixed-methods feasibility study" - Demonstrates the effectiveness of chatbots in providing asthma support through features like risk assessment and control tracking. - Reports a statistically significant 8% improvement in asthma control post-use of the chatbot. - Highlights the need for improved conversational depth and personalization to meet rising expectations for chatbot fluency. - Emphasizes the importance of balancing conversational capability with accuracy and safety in future health-focused chatbots. #chatgpt #asthma #chatbot #healthcare

AI tools, especially newer GenAI tools, can augment clinical diagnosis in so many ways - it can help patients seek care faster, it can improve communication between patients and their providers, it can also augment clinical decision-making to avoid premature closure and other such cognitive biases. On the other hand, these tools can also inadvertently exacerbate existing health disparities and result in patient harm. Getting this right is critical. Diagnostic errors are the number one cause of medical errors experienced by patients in the US, with 1 in 20 US adults experiencing a delayed or inaccurate diagnosis EVERY YEAR. Using new technologies to improve clinical diagnosis is a must, but we need to proceed with caution and vigilance. Join us on July 25th for a free, virtual webinar hosted by the The National Academies of Sciences, Engineering, and Medicine exploring the role of digital health and AI technologies to improve diagnosis. It's been a huge privilege serving as chair of an esteemed steering committee of national experts in clinical diagnosis and AI technologies - including Michael Howell, MD MPH, Julia Adler-Milstein, Judy Gichoya, Maia Hightower, MD, MPH, MBA, David Larson, Gene Harkless and Thomas Cudjoe, MD, MPH, MA. We've got an incredible line-up of speakers - including Michael Howell (Google), John Whyte (WebMD), Jonathan H. Chen (Stanford), Kadija Ferryman, PhD (Hopkins), Eric Horvitz (MSFT) and my boss Andy Bindman (Kaiser Permanente)! More info and registration can be found ⬇ Hope you can join us! UCSF Coordinating Center for Diagnostic Excellence (CoDEx)

139

7 Comments

👉🏼 Transcending Language Barriers: Can ChatGPT Be the Key to Enhancing Multilingual Accessibility in Healthcare? 🤓 Vaibhav Gulati 👇🏻 https://lnkd.in/eDkDyfA9 🔍 Focus on data insights: - ChatGPT's performance in simplifying and translating radiology reports into multiple languages - Comparison of accuracy and completeness in translated versions - Evaluation of medical term explanations in different languages 💡 Main outcomes and implications: - Spanish translation outperformed Hindi and Russian versions in certain categories - English version showed better performance in explaining medical terms - Need for further training for less commonly used languages in medical literature 📚 Field significance: - Potential for ChatGPT to improve multilingual accessibility in healthcare - Use of LLMs for translating and simplifying radiology reports - Impact on healthcare access and cost reduction 🗄️: [#multilingual #healthcare #translation #radiology #accessibility]

1

1 Comment