Tengyu Ma

OpenCoder: An Open-Source Code LLM Redefining Model Efficiency 🌟 The OpenCoder team has launched a game-changer in the world of code language models. Unlike many LLMs that focus on sheer size, OpenCoder brings something unique to the table: a data-focused training approach. By putting quality before volume, OpenCoder’s 1.5B and 8B models rival some of the biggest players, like DeepSeekCoder and QwenCoder, with only a fraction of their token count. The core strength of OpenCoder lies in its meticulous data processing pipeline and two-stage fine-tuning. With a training dataset composed of 2.5 trillion tokens, 90% raw code and 10% code-related web data, OpenCoder refines every aspect of its corpus. Advanced filtering and deduplication strategies, applied to 607 programming languages, optimize model efficiency and precision, ensuring high-quality data drives its training. Key Features: - Complete Training Transparency: Open-source datasets and training pipelines enable reproducibility and further research. - Instruction Fine-Tuning: Two-stage tuning sharpens both core CS concepts and real-world coding skills, prepping OpenCoder for production-level code generation. - Available on Ollama: Accessibility in deployment means more avenues for experimentation and improvement. OpenCoder's data-driven strategy showcases the potential of a leaner approach. By focusing on quality data rather than quantity alone, OpenCoder achieves file-level deduplication that outperforms traditional repo-level methods, making it both efficient and robust. 🚀 Read more here: https://lnkd.in/eEnG3yJA #OpenSource #MachineLearning #CodeLLM #AI #ModelEfficiency #DataQuality #Innovation #CodeGeneration #OpenCoder #TechNews #AIResearch #Programming #LargeLanguageModels #ArtificialIntelligence

1

Great to see the newly announced Bamba-9B, an inference-efficient Hybrid Mamba2 model 🐍 trained by IBM, Princeton, CMU, and UIUC on completely open data used PyTorch FSDP to train these novel architecture models and they are integrating inference with vLLM, which recently joined the PyTorch Ecosystem.

167

Check out the latest research on high-performance graph processing from a hardware/software co-design perspective! This paper dives deep into innovative strategies to optimize computational efficiency. https://lnkd.in/gUXumc5x 廖小飞 Hai Jin Long Zheng Zhiyuan Shao

This surprising paper by Catarina G. Belem et al., titled "From Single to Multi: How LLMs Hallucinate in Multi-Document Summarization," points out the fundamental problems of Perplexity and related AI search platforms. https://lnkd.in/et_D_Axs - LLMs struggle to identify shared information. The study found that when explicitly instructed to summarize only information shared across multiple documents, LLMs actually hallucinated more frequently. (LLMs tend to make up rumours - when prompted to prevent them) - Hallucinations increase when summarizing non-existent information and LLMs frequently generated fabricated summaries. This indicates a concerning tendency to invent information rather than recognizing its absence. Notably, GPT-3.5-Turbo and GPT-4 exhibited this behavior more often than open-source models. (try it yourself!) - Simple mitigation strategies are only moderately effective. The researchers tested various post-processing techniques to reduce hallucinations, including filtering out unrelated or redundant information. While these methods showed some promise, the improvements were generally small. (this argument remains open. I am actively engaged in advancing this field and encourage those interested to reach out)

2

Hey everyone ! Today LLM Paper, "Enhancing Large Language Models' Situated Faithfulness to External Contexts" by Yukun Huang, Sanxing Chen, Hongyi Cai, and Bhuwan Dhingra from Duke University, focuses on improving how large language models (LLMs) manage conflicting information from internal knowledge and external contexts. The concept they introduce is called situated faithfulness, where models balance the accuracy of their internal knowledge with external data, instead of relying too much on possibly unreliable external information. Large language models (LLMs) like GPT-4 often rely on external information to answer questions, but sometimes that information is just plain wrong. Think of it like someone feeding you facts, and you don't always know if they're telling the truth. The goal of this research is to make these models smarter about when to trust their own knowledge and when to trust what they're being told. They've come up with two main ways to do this: 1. Self-Guided Confidence Reasoning (SCR): The model basically asks itself, "Do I trust what I already know more, or should I go with this new info?" It weighs both sides before answering.    2. Rule-Based Confidence Reasoning (RCR): This method uses set rules to decide what to trust based on confidence levels, kind of like following a checklist. They test these methods using a bunch of question-answering datasets, including a brand new one they created called RedditQA, where they took real Reddit posts with wrong information to see how well models can handle misinformation. What’s new? - RedditQA is a fresh dataset with real-world incorrect info to test how well models can spot and ignore bad external data. - SCR vs. RCR: For big, smart models like GPT-4, the self-guided approach works best. But for smaller models, the rule-based approach actually does better. - They also created a fine-tuning technique called CR-DPO that improves how models handle this confidence reasoning, especially for smaller models, making them up to 8.9% more accurate. In short, the paper offers new ways to make AI models better at knowing when to trust themselves or external data, which is super important when the external info might be misleading or just plain wrong. Follow me for more insights: Bhoop Singh Gurjar 🇮🇳 daily AI news :https://lnkd.in/gzAV3HPH practical AI:https://lnkd.in/gwKADz9c weekly newsletter: https://lnkd.in/gFw6bwYs Blog :https://lnkd.in/gX57cqKT

5

A set of recent experiments with large language models (#LLM) such as #chatGPT suggest that using super-long prompts that contain up to hundreds or even thousands of examples outperforms fine-tuning. Given that fine-tuning can be a costly endeavor (although it is often a one-time exercise), it is exciting to see that packing a prompt with examples can be as good or better if the LLM you are using allows for large context windows. This approach, called in-context learning or ICL, trades "finetuning-time cost for increased inference-time compute". But I see an additional benefit of ICL, which is flexibility, as it alleviates the need for 'fine-retuning' if/when new or different examples become available. Maintaining and growing a library of examples, or as Ethan Mollick would say, a "grimoire" of shots, allows you to assemble any subset needed for a particular use case, the equivalent of having an infinite number of finetuned models at your finger tips. https://lnkd.in/gGaUmyEM

20

5 Comments

Ilya Sutskever confirmed at NeurIPS '24 pre-training scaling law will not be the same. Internet data, the AI fossil fuel, is running out. It is not the end of the AI hype, the rapid rise of foundational AI has outpaced our ability to fully grasp their long-term societal impact. AI agents and applications are poised to transform every facet of our lives, and we will see it happening in coming few years. There will still be improvements in model architecture, and synthetic data creation, so models will improve, just not at the lightning speed they did till now. Looking forward, the single most important milestone in AI is creating systems capable of advancing themselves—AI researchers that can innovate and improve foundational models autonomously. We need to assess how far are we from this stage.

9

Thanks Hamdi Amroun, PhD - this is a great paper to share. In the real world business context we often need AI to reason over large pieces of text, and often you need/want multi-document summarization (MDS). 'When testing five LLMs with benchmarks using news and conversation datasets, they found up to 75% of content in MDS summaries was hallucinated, with notable increases towards summaries' end. Alarmingly, even for non-existent topics, models like GPT-3.5-turbo and GPT-4 generated fabricated content 79% and 44% of the time, respectively. Analysis of 700+ generated insights showed that hallucinations often arose from failures to follow instructions or overly generic content.' I agree with Hamdi's sentiments that despite improvements in post-processing techniques, robust solutions are urgently needed to reduce hallucinations, as inaccurate summaries can lead to business risks (notably misinformation and misrepresentation). I would add that there is still a lot of utility in LLMs/Multimodal models but we need to be vigilant - this is not about AI malevolence or intentional deception. LLMs do not possess consciousness or intent; they generate content based on patterns in data they were trained on.

5

1 Comment

DSML trend: Fragility of LLMs in the real world In a new article published to the arXiv preprint database, MIT, Harvard and Cornell researchers found that Large language models (LLMs) like GPT-4 and Anthropic's Claude 3 Opus struggle to accurately model the real world, especially in dynamic environments. This fragility is highlighted when LLMs are used for navigation. Unexpected changes, such as detours or closed streets, can lead to significant drops in LLMs’ accuracy or total failure. LLMs trained on random data formed more accurate world models compared to those trained on strategic processes. This is possibly because random data exposes the models to a wider variety of possible steps, even if they are not optimal. The study raises concerns about deploying AI systems in real-world applications, such as driverless cars, where dynamic environments are common. The researcher warns that the lack of coherent world models in LLMs could lead to malfunctions. That’s my take on it: The disconnect between clean models and the messy real world is not a new problem. In fact, it mirrors existing challenges in conventional statistics. In parametric statistics, we often make unrealistic assumptions about data structures, such as normality and independence. Robustness to non-normality, heteroskedasticity, and other violations of these assumptions is a highly sought-after feature, and similar principles may apply to LLMs. We expect clean data, rely on linear models despite most real-world relationships being non-linear, and treat experimental methods as the gold standard. While controlled environments provide clarity and reproducibility, they often fail to capture the richness and unpredictability of real-world scenarios. Similarly, training LLMs on strategically optimized data may cause them to overfit to specific patterns, limiting their generalizability. A promising approach to address this challenge could be to combine LLMs with other models, such as reinforcement learning agents trained in dynamic simulations, to enhance their understanding of complex and dynamic environments. Link: https://lnkd.in/g9Bz9p-n

1

More happy news: our second (and last) accepted paper for #NeurIPS 2024: "Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities" We present a method for uncertainty quantification for LLMs. Briefly, for a given input, generate multiple outputs, cluster them, and calculate the corresponding von Neumann entropy. Unlike previous methods, this takes into account dependencies between the clusters and leads to more fine-grained uncertainty estimates. Work done together with Alexander Nikitin and collaborators from Oxford (Jannik Kossen, Yarin Gal). Preprint: https://lnkd.in/d38F3X4e

79

1 Comment

1. 🚀 TextGrad uses text feedback for automatic differentiation, offering a new method for prompt optimization in large language models, effective in Q&A, molecular optimization, and radiotherapy planning. 2. 🔧 DSPy provides the algorithmic foundation for TextGrad to optimize LLMs' prompts and weights, reducing the need for manual adjustments. 3. 🌐 Automatic differentiation is crucial in deep learning, and TextGrad applies it to the text domain, optimizing through text feedback. 4. ✨ ProTeGi leverages natural language gradients and prompt editing to effectively optimize LLMs' prompts, finding the optimal configuration. 5. ❓ Despite TextGrad's innovation, its practical application may be limited by the performance of the teacher model and the need for extensive API calls. #AI #TextGrad #Innovation #DSPy #LLMOptimization #Algorithm #AutoDiff #DeepLearning #ProTeGi #PromptOptimization #AIOptimization #AIChallenges #TechLimitations

1

LLMs are powerful but prone to 'hallucinations'—false yet plausible outputs. In our #NeurIPS2024 paper, we introduce a compute-efficient method for detecting hallucinations in single responses using hidden states, attention maps, and output probabilities. Our approach achieves >45x speedups over traditional methods while significantly boosting detection accuracy across diverse datasets. Link to the paper: https://lnkd.in/eBWNXRkx Joint work with Gaurang Sriramanan Siddhant Bharti Vinu Sankar Sadasivan Shoumik Saha Priyatham Kattakinda

309

2 Comments

🌟 Exploring ViSNet in the Recently Published AI2BMD Neural Network Potential 🌟 I recently came across the new AI2BMD neural network potential just published in Nature, and its underlying architecture—ViSNet—really stood out. ViSNet offers a unique approach to molecular modeling by efficiently capturing molecular geometry—bond lengths, bond angles, and dihedral angles—using innovative, low-complexity vector operations. This design strikes a strong balance between detail and efficiency, making it ideal for applications in molecular dynamics. 🚀 Central to ViSNet’s design is the Runtime Geometry Calculation (RGC) module, which dynamically extracts high-order geometric features as needed, avoiding explicit tensor storage. The key idea here is to break down complex geometric features into manageable vector representations. For instance, bond lengths are derived directly from the distance between atoms, while bond angles and dihedral angles are calculated from vector inner products and rejections between neighboring atoms. By summing unit vectors between atoms, RGC efficiently captures angular and torsional information at linear complexity, avoiding the O(N³) cost of traditional tensor operations. This keeps computational demands low and allows ViSNet to scale effectively to larger molecular systems, even when full geometric detail is required. 🧬 Vector-scalar interactive message-passing (ViS-MP) mechanism is another key method. Instead of treating scalar and vector features independently, ViSNet integrates them within each layer, letting vector features (like directions) and scalar features (like bond distances) inform each other. This combined representation captures detailed spatial information with fewer samples, enhancing data efficiency without sacrificing accuracy. Benchmarking on datasets like QM9, MD17, and Molecule3D, ViSNet demonstrated strong performance in predicting molecular properties such as energy and atomic forces, achieving high-fidelity results with as few as 950 training samples. 👍 The AI2BMD paper shows how ViSNet’s careful balance of accuracy, computational efficiency, and scalability can support realistic simulations, driving forward practical applications in drug discovery and beyond. 🌱 👉 Check out the ViSNet paper: https://lnkd.in/ggUZAeVg 👉 Check out the AI2BMD paper: https://lnkd.in/gxR6wtZX #DrugDiscovery #MolecularModeling #MachineLearning #NeuralNetworkPotential #AI4Science

32

As we have always argued at samvid.ai , merely making chunks of a document available as reference to an LLM cannot make answers consistent even with temperature set to zero. The only solution is an expertise layer (our very own sootras and neeyums) that can rein in the LLM to focus on the fundamental essence of the document. This enables samvid.ai to deliver latent knowledge to decision makers correctly, concisely and consistently. Reach out to me for a demo today! #enterpriseai #responsibleai #expertsystems #sootras #neeyums https://lnkd.in/gh46qqcB

37

One new interesting application of LLMs: CAIS has released a forecasting bot that provides probability estimates for any question you ask. For example, to the question "Will GPT5 be released before 2025?", it answers that it's 70% likely. The bot works by searching for relevant information online, feeding it to GPT-4o, asking a few guiding questions, and then requesting a probability estimate. What's impressive is that this bot outperforms experienced human forecasters, even though GPT-4o wasn't specifically trained for this task. This kind of application shows that AI could be of great help in improving decision-making in the future. Link to the blog post: https://lnkd.in/gjP7JQgz Link the bot if you want to try it yourself: https://forecast.safe.ai/

11

*The* hackathon of the summer is back! The UC Berkeley AI Hackathon, organized by SkyDeck and Hackathons @ Berkeley, is June 22-23, and we just opened applications. Last year, over 1200 students, grad students, and new grads attended. And we can’t wait to top it this year! So, mark your calendars, and apply for the one-of-a-kind event now! For 36 hours, hackers will have privileged access to the invite-only OpenAI API (including GPT-4 access), plugins, and some of the best large language model (LLM) hacking tools on the planet. And there is a chance to win some of the $100,000 prize money from the Berkeley SkyDeck Fund for your project. Priority applications close on June 1st @ 11:59 PM Apply at apply.hackberkeley.org. Events like this Hackathon are what help to make Berkeley the biggest leader in innovation! Can’t wait to see the innovation that comes out of this year!

271

11 Comments

𝐌𝐮𝐬𝐭-𝐫𝐞𝐚𝐝, 𝐟𝐫𝐞𝐞 𝐛𝐨𝐨𝐤 𝐭𝐡𝐚𝐭 𝐩𝐫𝐨𝐯𝐢𝐝𝐞𝐬 𝐚 𝐩𝐚𝐭𝐡𝐰𝐚𝐲 𝐟𝐫𝐨𝐦 𝐆𝐀𝐈 𝐭𝐨 𝐀𝐆𝐈! ​ Yann LeCun has expressed skepticism about the potential of LLMs to achieve Artificial General Intelligence (AGI), citing limitations in memory, planning, and grounding in real-world understanding. However, we argue that 𝗟𝗟𝗠 𝗖𝗼𝗹𝗹𝗮𝗯𝗼𝗿𝗮𝘁𝗶𝘃𝗲 𝗜𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲 (𝗟𝗖𝗜)—particularly with multiple multimodal LLMs working together—offers a promising architecture to address these challenges and progress toward AGI. ​ Two key architectural innovations underlie LCI: ​ 𝐒𝐞𝐩𝐚𝐫𝐚𝐭𝐢𝐨𝐧 𝐨𝐟 𝐔𝐧𝐜𝐨𝐧𝐬𝐜𝐢𝐨𝐮𝐬 𝐏𝐫𝐨𝐜𝐞𝐬𝐬𝐢𝐧𝐠 𝐚𝐧𝐝 𝐂𝐨𝐧𝐬𝐜𝐢𝐨𝐮𝐬 𝐑𝐞𝐚𝐬𝐨𝐧𝐢𝐧𝐠 𝐚𝐧𝐝 𝐏𝐥𝐚𝐧𝐧𝐢𝐧𝐠: Inspired by the human mind’s dual-layer structure, LCI divides its architecture into foundational and adaptive layers. The foundation layer, akin to unconscious processing, leverages extensive datasets to build robust pattern recognition, encoding essential responses much like human instincts. The adaptive layer, mirroring conscious thought, enables rapid adaptation and contextual reasoning. This dual-layer approach allows LLMs to perform complex tasks with foundational knowledge while adjusting to new information on the fly, much as a child learns new concepts within a developed cognitive framework. ​ 𝐃𝐢𝐬𝐭𝐢𝐧𝐜𝐭 𝐑𝐨𝐥𝐞𝐬 𝐟𝐨𝐫 𝐄𝐚𝐜𝐡 𝐋𝐋𝐌-𝐀𝐠𝐞𝐧𝐭: Each LLM-agent is assigned a specific role, such as executive (knowledge processing), legislative (behavior guardrails), and judicial (adapting to cultural norms). ​ Additionally, our validation of the LCI framework has been thoroughly tested through real-world deployment in applications across various domains, including healthcare, sales planning, investment, and debiasing news. These empirical deployments highlight the practical viability and adaptability of the LCI framework, showcasing its ability to enhance reasoning and decision-making through collaborative intelligence. ​ The eleven aphorisms presented in this free book illustrate the philosophical foundation of the LCI framework, backed by empirical studies. Through LCI, we envision a clear pathway to AGI, rooted in collaborative intelligence and flexible, context-aware adaptation. https://lnkd.in/gn74DkNj (free) https://lnkd.in/gszSvfqP ($6.98 printing fee)

9

New GAI Textbook Announcement (The textbook is being used by Stanford CS372: AI for precision medicine and psychological disorders) The past two years have been a significant period in the field of AI, during which we have observed substantial strides in generative AI technologies. Having dedicated over 5,000 hours to research and analysis, I have compiled this book to share a collection of findings and hypotheses regarding Large Language Models (LLMs)—detailing their operational mechanisms, and strategies to mitigate biases and reduce instances of hallucination. The central premise is to enhance context awareness and define human objectives with greater precision. By doing so, LLMs can more effectively mirror human linguistic behaviors, utilizing context-specific linguistic features to fulfill intended goals. SocraSynth: Socratic Synthesis with Multiple Large Language Models. https://lnkd.in/giC4474p

25