Curtis Langlotz

#AI is fundamentally going to change human history The Fitz-Gerald Group principal Keith Fitz-Gerald analyzes Nvidia and unveils his top quantum computing stock on 'Making Money.' https://lnkd.in/ee-f3TNT

1

Join us on June 5, 2024, at 1 PM (ET) for a webinar on tackling the reproducibility and generalizability challenges in #MachineLearning ! Learn strategies for data accessibility, leakage prevention, preprocessing, algorithm selection, and more to ensure reliable ML outcomes. Farhad Maleki Society for Imaging Informatics in Medicine (SIIM) Register: ⬇ https://lnkd.in/dqrDUg2J Objectives: - Equip participants with strategies to facilitate data accessibility, prevent data leakage, and standardize data preprocessing to enhance the reproducibility of machine learning projects. - Provide insights into selecting appropriate algorithms, promoting code sharing, and managing dependencies effectively to achieve reliable machine learning outcomes. - Advocate for the adoption of comprehensive and transparent research documentation standards to improve the clarity and reproducibility of machine learning research. - Demonstrate how to utilize advanced tools and frameworks and adhere to community standards to significantly improve the reproducibility of machine learning systems. #ML #DataScience #Healthcare #Webinar #AI

14

In our hospital we deployed AI using a PDSA framework similar to this, but this infographic also comes with a link to published manuscripts, which is very handy. Lunit Cancer Screening DeepTek CARPL - Radiology AI Platform Synapxe

14

Generative artificial intelligence is infiltrating peer review process 🤖 Navigating the New Frontiers: Generative AI in the Peer Review Process As the influx of manuscript submissions grows, the peer review process in scientific research faces increasing challenges. Recent advancements have seen generative AI tools like #ChatGPT being tested to support this crucial aspect of publication. However, their integration raises several ethical and practical concerns, including confidentiality and the preservation of critical human judgment. This recent study highlighted in Critical Care explores these dynamics, demonstrating both the potential efficiencies brought by AI and the complexities it introduces. What are your thoughts on this development? Can AI truly complement the human element in peer review without compromising the integrity and thoroughness required in scholarly publishing? #PeerReview #ArtificialIntelligence #ScientificPublishing #EthicsInAI Artificial IntelliGence in Anestesia e Terapia IntensivA Elena Giovanna Bignami Marco Cascella Jonathan Montomoli Valentina Bellini

1

See Current Standings: 👆🏽 UPDATE 5 of 6 of the OCI Foundation's VLSS SOCIAL MEDIA CONTEST (Correct as of 11.08 AM, 04/05/2024; Nigerian Time). Just over 24 hours to go: KWARA (96%), GOMBE (90.0%; 7,123) and BAUCHI (85.0%) maintain the top 3, and any of them can still finish top!!! We wait to see who has the guts to push through to the final day . . . Any of the 9 states can still finish in the cash-winning top 3. IMPORTANT NOTES: 1. To Vote, follow this link: https://lnkd.in/eTXQyvYh. 2. A balanced approach on all VLSS components gives more points than a very high score in one component. 3. We strongly recommend that contestants should target all of the Views (20%), Like (20%), Share (30%) and Subscribe (30%), not just the views. 4. To win easily, see where the points are, and watch some winning strategies here: https://lnkd.in/gJB8v6EF. 5. This all ends at midnight on Saturday, May 4, 2024. Results will be announced within 48 hours after that, to allow for updates on the Subscription Scores (data on this lags by 48 hours from TikTok) All others (Views, Likes and Shares) will be frozen at this time. 6. A minimum of 60% VLSS Percentage Score AND a Top 3 finish are needed for the main 50k rewards. 7. Any amount won by any State will be shared at the ratio of 50% to the participating Corps Member(s) and 50% to the Anchors(s). 8. If there is a tie in the PERCENTAGE VLSS SCORE, the RAW VLSS SCORES are used to separate contestants. #werisebyliftingothers #ocifoundation

1

Rewriting the Rulebook of Healthcare Education in the Age of AI In our latest article, “Preparing Healthcare Education for an AI-Augmented Future” (Jiajie Zhang & Susan Fenton, 2024), just published in npj Health Systems, we deliver a wake-up call: AI isn’t just changing healthcare—it’s rewriting the entire rulebook of education and practice. It’s liberating us from cognitive labor, demanding that we redefine how we learn, teach, and solve problems. This is no gradual shift. It’s a seismic transformation reshaping humanity’s relationship with knowledge - a revolution on par with the Agricultural and Industrial Revolutions. Key Ideas for Radical Change: •   AI as Knowledge Systems Redefined: AI systems aren’t just databases—they are dynamic, supercharged knowledge engines, trained on nearly all of humanity’s collective knowledge. They don’t just store; they think, performing advanced cognitive tasks once reserved for humans. •   Distributed Cognition is the New Reality: The era of isolated learning is over. Knowledge now spans across the relatively static human brain and AI systems, which are growing at an explosive speed. Educators must stop resisting and start equipping learners to thrive in this new paradigm of AI-human collaboration. •   No More Silos—Welcome Unified Knowledge: AI, as a system with integrated and unified knowledge, demolishes the boundaries between disciplines. Healthcare professionals can finally escape the constraints of overspecialization and embrace a unified, dynamic knowledge base to tackle problems holistically. •   Education Redefined: Memorization? A relic of the past. Tomorrow’s healthcare education demands creativity, ethical reasoning, problem-solving in real-world contexts, and mastering the art of collaborating with AI. Anything less is a failure to prepare for the future. Why This Matters: This is the Cognitive Revolution. It’s reshaping how we innovate, how we care for patients, and how societies function. But this transformation will only succeed if healthcare educators, leaders, and accrediting bodies take decisive action. We must urgently revise curricula to embed AI at the core of learning, ensuring healthcare professionals are equipped to navigate an AI-augmented world with competence and confidence. Let’s Shape the Future Together: How do we integrate AI into our healthcare systems and education programs effectively? How do we ensure that ethical, human-centered care remains at the heart of this revolution? Share your ideas, challenges, and visions for the future. #AI #education #accreditation #Healthcare #CognitiveRevolution #FutureOfEducation #MedicalEducation #DistributedCognition #AMIA #HIMSS #txhimss #informatics #datascience #healthsystems McWilliams School of Biomedical Informatics The University of Texas Health Science Center at Houston (UTHealth Houston) The University of Texas System Nature Magazine Springer Nature AMIA (American Medical Informatics Association) HIMSS Houston HIMSS

48

The FDA’s latest Executive Summary highlights a significant evolution in regulatory thinking for generative AI (#GenAI) devices. While previous guidance addressed AI broadly, this new approach focuses on the unique complexities of GenAI: + Foundation Models: Acknowledges challenges in managing and regulating third-party models that power GenAI, emphasizing the need for transparency and adaptability. + Non-Deterministic Outputs: Spotlights risks like "hallucinations," requiring stricter controls and tailored performance metrics. + Postmarket Monitoring: Proposes new strategies to address GenAI’s continuous learning and real-world variability. This marks a shift toward a more dynamic, lifecycle-based approach to ensure safety. What are the implications for entrepreneurs? Transparency is crucial—entrepreneurs must document their model's design, training data, and performance metrics, even when leveraging third-party foundation models. Narrowly defined use cases are equally important to ensure products are safely and effectively evaluated, mindful of the competing pressure to create applications with broad utility (and market potential). Startups should prioritize robust monitoring systems to track real-world performance, mitigate risks like bias or hallucinations, and maintain consistent outputs across diverse deployments. As always, human-in-the-loop safeguards and transparent, user-centric designs can enhance trust and safety. FDA’s lifecycle approach means compliance doesn’t stop at approval—continuous evaluation, updates, and monitoring will be increasingly expected to keep products aligned with regulatory and patient safety standards. Explore the full document for insights: https://lnkd.in/dXefRaaS #wearenina #healthcaretechnologies Nina Capital

79

5 Comments

📱As we embrace artificial intelligence (AI) to transform healthcare, it’s crucial to recognize the importance of a structured approach to evaluating #AI tools for healthcare organizations prioritizing transparency, accountability, and inclusivity. The 𝐎𝐏𝐓𝐈𝐂𝐀 tool provides a structured framework for evaluating AI tools in healthcare, with a focus on key points that healthcare organizations should consider before adopting these technologies. ⬇️ Here are the 🔑 main takeaways: 🔸 𝐂𝐥𝐢𝐧𝐢𝐜𝐚𝐥 𝐑𝐞𝐥𝐞𝐯𝐚𝐧𝐜𝐞: AI tools should address well-defined clinical problems and offer solutions that clearly improve patient care. The clinical problem must be significant and the AI solution should align with the organization’s strategic goals. 🔹 𝐕𝐚𝐥𝐢𝐝𝐚𝐭𝐢𝐨𝐧 𝐢𝐧 𝐑𝐞𝐚𝐥-𝐖𝐨𝐫𝐥𝐝 𝐒𝐞𝐭𝐭𝐢𝐧𝐠𝐬: AI models should not only show promise in controlled environments but must be tested in real-world #healthcare settings. This ensures that the AI tool can handle the complexity and variability of actual patient populations. 🔸 𝐂𝐨𝐧𝐭𝐢𝐧𝐮𝐨𝐮𝐬 𝐌𝐨𝐧𝐢𝐭𝐨𝐫𝐢𝐧𝐠: Once deployed, AI tools need ongoing monitoring to assess their performance over time. Changes in clinical practices, patient demographics, or data collection methods can affect the performance of AI models, necessitating updates and recalibration. 🔹 𝐈𝐧𝐭𝐞𝐠𝐫𝐚𝐭𝐢𝐨𝐧 𝐢𝐧𝐭𝐨 𝐖𝐨𝐫𝐤𝐟𝐥𝐨𝐰𝐬: AI tools must be seamlessly integrated into existing #clinical workflows. Tools that disrupt current practices without providing significant benefits will face resistance from healthcare professionals. 🔸 𝐄𝐭𝐡𝐢𝐜𝐚𝐥 𝐂𝐨𝐧𝐬𝐢𝐝𝐞𝐫𝐚𝐭𝐢𝐨𝐧𝐬: It’s important to consider the ethical implications of AI use, such as biases in #data, algorithmic transparency, and patient #privacy. AI tools must be evaluated for their potential to inadvertently reinforce healthcare disparities. 🔹 𝐄𝐜𝐨𝐧𝐨𝐦𝐢𝐜 𝐕𝐚𝐥𝐮𝐞: Healthcare organizations should assess the cost-effectiveness of AI tools, weighing the benefits in improved outcomes and efficiency against the costs of deployment, training, and maintenance. 🔸 𝐑𝐞𝐠𝐮𝐥𝐚𝐭𝐨𝐫𝐲 & 𝐋𝐞𝐠𝐚𝐥 𝐎𝐯𝐞𝐫𝐬𝐢𝐠𝐡𝐭: AI tools should comply with regulatory standards and undergo rigorous review to ensure safety and efficacy. Compliance with legal and #ethical guidelines is crucial, especially when dealing with patient data. ❗️It is critical we thoughtfully integrate AI into our clinical practice, ensuring that the tools truly enhance clinical care, are safe, and deliver value both to patients and the healthcare system (clinicians/payors/hospitals). #quality #healthcarequality #patientsafety #medicine #artificialintelligence NEJM AI American College of Cardiology American Society of Echocardiography American Heart Association Harborview Medical Center University of Washington UW Medicine + Heart Institute Noa Dagan Stav Devons-Sberro Dr. Ziv Paz Ran Balicer https://lnkd.in/gxEEx2Vp

24

4 Comments

Qualitative metrics from the biomedical literature for evaluating large language models in clinical decision-making: a narrative review - Most frequently used LLMs include GPT-3.5, GPT-4, Bard, LLaMa/Alpaca-based models, and Bing Chat. - Key evaluation criteria for LLM outputs are accuracy, completeness, appropriateness, insight, and consistency. - There is significant variation in how studies report findings and assess LLM performance; standardized reporting metrics are needed. DOI: https://lnkd.in/e3iikhzJ DOI: doi:10.1186/s12911-024-02757-z

1

Our recent study demonstrates the potential of diffusion probabilistic models for generating real-world synthetic electronic health records (EHRs) for emergency departments. We found that diffusion models are more computationally efficient than generative adversarial networks (GANs) and perform better in terms of data similarity, privacy protection, and utility. The synthetic data generated is statistically comparable to real EHR data, making it highly valuable for machine learning applications. https://lnkd.in/gqwJRgbi

27

This paper evaluates the accuracy of citations provided by LLMs in response to medical questions. 1️⃣ Developed an automated evaluation pipeline to assess the relevance of sources provided by LLMs for medical statements, validated by 88% agreement with medical experts. 2️⃣ Evaluated five LLMs and found that 50-90% of their medical responses were not fully supported by the cited sources. GPT-4 with retrieval-augmented generation (RAG) still had 30% unsupported individual statements. 3️⃣ Many URLs generated by LLMs were invalid, contributing to citation issues. 4️⃣ The study involved a panel of medical experts to validate the automated evaluation framework. 5️⃣ Created and open-sourced a dataset of 1200 medical questions and expert annotations. Metrics include URL validity, statement-level support, and response-level support. 6️⃣ Found significant variability in the quality of sources provided, with response-level support ranging from 7% to 54% across models. 7️⃣ LLMs performed worse on questions from user-generated content compared to vetted medical sources, highlighting the challenge of providing reliable medical information from diverse sources. ✍🏻 Kevin Wu, Eric Wu, Ally Casasola, Angela Zhang, Kevin Wei, Teresa Nguyen, Sith Riantawan, Patricia Shi, Daniel Ho, James Zou. arXiv. February 2024. DOI: 2402.02008v1

135

12 Comments

👉🏼 Qualitative metrics from the biomedical literature for evaluating large language models in clinical decision-making: a narrative review 🤓 Cindy N Ho 👇🏻 https://lnkd.in/etSsfP28 🔍 Focus on data insights: - 📊 The literature review identified 108 relevant articles that assessed LLMs in clinical contexts, showcasing significant engagement in this emerging field. - 🧠 The most utilized LLMs included GPT-3.5, GPT-4, Bard, and other models, indicating a trend towards specific tools within medical settings. - ✅ Five key criteria emerged for evaluating LLM outputs: "accuracy", "completeness", "appropriateness", "insight", and "consistency", which may streamline evaluation processes. 💡 Main outcomes and implications: - 📈 A consistent identification of high-quality LLM attributes emphasizes the need for standardized assessment tools in medical AI applications. - 🔄 The observed variability in performance reporting suggests that researchers should aim for more uniform methodologies to enhance comparability across studies. - 🛠️ Development of standardized qualitative metrics is crucial for advancing research and fostering trust in LLMs for clinical decision-making. 📚 Field significance: - 🌟 This research highlights the integration of advanced AI technologies in healthcare, marking a pivotal step towards improving clinical support systems. - ⚖️ Establishing clear evaluation standards can lead to better regulatory practices and guidelines for deploying LLMs in medical environments. - 💡 Understanding the effectiveness of LLM outputs can transform patient care through more accurate and timely decision-making processes. 🗄️: [#largeLanguageModels] [#clinicalDecisionMaking] [#healthcareAI] [#evaluationMetrics] [#standardization]

2