AI for medical coding by U.S. researchers

This makes complete sense as many diagnosis codes require piecing together multiple pieces of information for a nuanced diagnosis, rather than a quick generic code. So having this result makes sense since the majority of data large language models are trained on by default would lean towards generic higher level diagnosis codes along with few of them, influenced by the dominance of fee-for-service models in healthcare. And though MA has been a leader in pushing for more nuanced coding to understand unique patient populations better, it's a smaller unique data set and has issues. And beyond just diagnosis codes, billing for additional codes to convey crucial information through claims is challenging due to the complexities and costs of exchanging EHR data, especially for small independent physicians. While large language models have a role in healthcare, overreliance without human expertise to help guide the LLM or innovative datasets may lead to much longer learning timelines and potentially harmful practices just being continued. It's vital to balance AI with human knowledge to avoid negative impacts on patient care and treatment protocols. And it is also important to think outside the standard approach grab all current data only to make sure we don't build in current biases that might be hiding fascinating finds that benefit massive amounts of people. We have already seen innovative approaches breaking traditional healthcare approaches through AI results and that is improving so many lives. Data is a powerful tool for shaping our future. Responsible use and creative approaches are essential for leveraging data effectively.

Study Finds LLM Errors in Clinical Coding "Unacceptably Large" A new study from the Icahn School of Medicine at Mount Sinai has found that currently available (LLMs) like GPT-3.5, GPT-4, Gemini Pro and Llama2-70b are not sufficiently accurate to automate the assignment of medical codes for billing and research purposes. The researchers compared these state-of-the-art LLMs on their ability to match over 27,000 diagnosis and procedure codes to their official text descriptions from Mount Sinai patient records. They found that all models scored below 50% accuracy, with GPT-4 performing best at around 45% for diagnosis codes and 50% for procedure codes. The researchers determined the extent of errors was still "unacceptably large" for using LLMs to fully automate medical coding without human oversight. What does this mean? General-purpose LLMs have limitations for specialized healthcare tasks like medical coding. There is a need for more specialized, tailored AI models and tools specifically designed and refined for healthcare use cases. Integrating LLMs with expert clinical knowledge could improve their accuracy. Rigorous evaluation and iterative refinement will be critical before deploying AI The implications point toward a future where AI will likely play an increasingly important role in healthcare, but via specialized models that are thoroughly validated. General AI may augment human expertise, but is unlikely to fully automate complex clinical tasks anytime soon without significant refinement tailored to the healthcare domain. More like this? I invite you to my LinkedIn group, “Artificial Intelligence in Mental Health”—a vetted, science-based forum focused on the intersection of AI and mental health care, free from promotions or marketing. Link in comments. #ai #aihealthcare #healthcare #coding #LLM The Study: https://lnkd.in/gmaf_4Z8

As Medicare Advantage organizations adopt and integrate #AI into their medical coding processes, here's a great guide to how it can help, and where its limitations lie: https://lnkd.in/dw-pathA #medicareadvantage #riskadjustment

Is this NEJM AI article a crushing indictment of LLMs' capabilities with regard to clinical documentation and medical coding? Are we still years away from this technology being practically deployable in patient care and hospital operations? At first, the headline and subsequent reporting about this paper would suggest as much, but looking into the methods of this paper demonstrates a subtler message. To start, I applaud the authors' efforts to cut through some of the hype around LLMs and show that they can't perform complex clinical reasoning right "out of the box" (at least to a degree that's ready for production). Taking a deeper dive, however, it's very important to note that their approach does not include any elements of fine-tuning or the use of RAG (Retrieval-Augmented Generation), which would be essential to getting reasonable results for a task as rule-based and outlier-heavy as medical coding. If anything, the study demonstrates to me that LLMs match concepts quite accurately in many cases, and refinement with already-existing tools is likely to improve performance dramatically. Capabilities to add content for RAG weren't present in OpenAI's ChatGPT at the time of this paper, but were added shortly thereafter, as I recall. Would love to see this study repeated to include the corpus of coding rules as a vector database, as I suspect even in a few short months, the results of this study would be dramatically different. One takeaway is that AI is moving faster than the traditional research enterprise can easily adjust to, and we'll need new paradigms to examine these models and ensure that patients benefit from these technologies as soon as they are safe for deployment. However, it's easy to criticize and speculate, so perhaps the most important takeaway is that I'm delighted we have great researchers like the team in this article starting to tackle these important practical issues. Would love thoughts from others in my network who are far more expert than me! #LLM #AI #CDI #HIM #MedicalCoding

As Medicare Advantage organizations adopt and integrate #AI into their medical coding processes, here's a great guide to how it can help, and where its limitations lie: https://lnkd.in/g45XNbEw #medicareadvantage #riskadjustment

🗞 NEJM AI Large Language Models Are Poor Medical Coders — Benchmarking of Medical Code Querying New models may perform better, but there's still work to do 😊! Read more: https://lnkd.in/eEjn6JZ5 #MedicalAI #LLM #HealthcareInnovation

This may only further improve further and disrupt medical billing and coding. With more industry participation in democratizing and anonymizing handwritten clinical notes from various EMR systems AI would not only improve coding accuracy but may also evolve to customize how a provider system can maximize revenue coding for all the services provided. #aiinhealthcare

#ai models are terrible #medicalbilling coders without a lot of work. This study from the NEJM Group, published in January, took 12 months of data from a reputable EHR and attempted to generate medical billing codes using clinical notes. They compared against GPT-3.5, GPT-4, Gemini Pro, and Llama2-70b. Results? GPT-4 had the highest exact match rate: 33.9% of the time it got the right ICD-10 code, and 49.8% of the time it got the right CPT code (oh, and 45.9% of the time it got the right ICD-9 code--which is an historical code now). Of course, there are a lot of caveats--no advanced "prompt engineering", no optimization, and of course the models have gotten better. The speculation here is that clinical notes are very long and difficult for LLMs to focus, leading to higher error rates.

🚀 Exciting blog article from Qantev AI Research Team! 🚀 We've just published the latest installment in our blog series on the transformative power of AI in medical coding. In this deep dive, we continue our exploration of the burgeoning field of automated medical coding, comparing traditional supervised models like PLM-ICD with innovative strategies leveraging Generative AI and prompt engineering. Our findings discuss the significant advancements and potential benefits of using foundational Large Language Models (LLMs) for medical code inference, especially in handling rare codes and navigating complex ICD ontologies without specialized tuning. This approach not only improves accuracy but also enhances efficiency in medical coding processes. Discover how Qantev is at the forefront of this technology, shaping the future of health insurance operations with AI-driven solutions. We're not just following the trends—we're creating them! 🔗 https://lnkd.in/eHADbppc #AI #HealthTech #MachineLearning #MedicalCoding #Innovation #healthinsurance #healthclaims

Very proud to post our team's work building AI that can draw insights from medical records. https://lnkd.in/gSi72uEs A few takeaways that stuck out to me from the data: 1. It's easy for LLMs to learn medical knowledge, but getting them to use it -- i.e. to understand and answer a question -- requires a layer of intelligence that is much harder and only the best models can do it (beyond our model "LLMD", kudos to GPT-4o) 2. Asking an LLM to interpret medical records is a step further because records are written in their own language that differs doctor to doctor, facility to facility, etc. And beyond that, each record is just a narrow slice of your healthcare journey, so you need to look at lots of them together to have an accurate picture of someone's health. 3. But LLMs and today's AI can do it (when trained properly) and the results are astonishingly useful! -- read the paper for details, and keep an eye out for PicnicHealth's patient-facing release this November to see for yourself.