Conor Kelly

Is Your RAG a Step Backwards? Excellent tutorial (below) from Haziqa Sajid in the all-important field of hybrid search—combining dense vectors with other field data types. Anyone with existing search configurations will know that hybrid approaches are a necessity and can be very challenging compared to merely building some greenfield RAG demo. In my experience, the real challenge is in ranking, which is the core of search. Many folks implementing RAG are entering the world of search, albeit via generative approaches. The uninitiated are quickly finding out that relevance is everything. Moreover, they’re discovering an even more pressing issue: user intolerance of irrelevant or noisy results, which can easily kill the entire effort—users simply stop using the shiny new toys and revert back to legacy systems, many of which are sufficiently well optimized. Compared to such systems, naive implementations of LLM-backed solutions can seem like a backward step.

1

1 Comment

Nature has an article "Scalable watermarking for identifying large language model outputs" https://lnkd.in/eE6uhrkD. #generativeai #watermark #largelanguagemodels #nature #research #artificialintelligence

3

This week, as my storage punditry task, I explore the role(s) SNIA #ComputationalStorage can play in creating #Data value for various applications in the data pipeline, including #AI. #DataTaxonomy is part of metadata, data tagging, and data enrichment that leads to #DataQuality. What better way to do this at the data source, the ingestion point. Check out my blog https://wp.me/p2fCSl-242

5

This GA includes new features including using local Tables, UDFs and Procs in your local Snowpark tests which is impressive, useful and I found my test suites running in as little as 0.11 seconds. https://lnkd.in/eitXx77T I think this is the only way to do local testing using identical code that works when deployed to Snowflake Cloud? #snowflake #snowpark #pytest

6

#Multimodality is quickly moving from an impressive demo feature to a basic expectation in apps harnessing #GenAI. ICYMI, here's a good article by Manoranjan Rajguru on how to apply you RAG solution augmented with Vision capabilities over Hierarchical document structures. https://lnkd.in/gxcDHguw

20

2 Comments

Fascinating to me how engineering innovation at the tooling level is equally magical to and keeping up with the LLM frontier model innovation! Super exciting progress with the #Azure #AI Model Inference API as a unified API to awesome models from Cohere, Meta, Mistral AI, Core42 and more without having to change orchestration code while integrating with Azure GenAI Gateway within API management for a scalable architecture. https://lnkd.in/ghutPvBp

28

The decision to ban “AI” generated code in operating systems makes a lot of sense to me. Think about it. If you were one of the members of a thesis defense committee would you accept one that was written with “AI”? What about accepting an evaluation report on a critical component? Or the answers to an online course? Is this was #AI supposed to be doing? Really? #artificialIntelligence #spe #machineLearning #dataScience

41

3 Comments

As the AI & Data team at Microsoft, we spend every day with clients exploring the potential of Generative AI. Our schedule is packed with Hackathons, Executive Briefings, and workshops at the Microsoft Technology Centre. It’s an exciting time as we see more and more customers deploy AI at scale and in production. Last night's earnings demonstrate remarkable growth, reflecting the incredible demand for Azure, the World’s AI Supercomputer    📈 Cloud and AI Surge: Microsoft Cloud exceeded expectations with $35 billion in revenue, marking a 23% increase. Azure's performance was stellar, contributing significantly to this success.  🤖 AI at the Forefront: The integration of AI across our products is transforming industries. More than 65% of the Fortune 500 now use Azure OpenAI Service, leveraging our cutting-edge AI accelerators and models.  🌍 Future Investments: Looking ahead, we are ramping up our investments dramatically, planning for a monumental scale-up in our cloud and AI infrastructure to continue leading the tech revolution. I have never worked in an environment that is moving so fast. The future is bright, and we're just getting started! #Microsoft #AI #CloudComputing #TechInnovation #Leadership #openai Hussain Mahmood Christian Schaetz Luca Callegari Sascha Utzen Edip Saliba Jutta Heinel Marcel Franke Hendrik Juelich Tosca Maria Colangeli Nevenka Scott Petra Pirron Lina Schröder Benjamin Kirnich Florian Deter

105

This is super impressive, and if it can actually support COBOL, it's a game changer for a couple of industry and government agencies that still use COBOL - a big issue because you literally can't find anyone who knows those old languages anymore. The rate improvements for these programming-focused LLMs are incredible and are going to put massive pricing pressure on code assistant copilots. Granite is interesting also because these are relatively small models - much cheaper to run in-house with low latency. As the hourly rental cost of AI accelerators HW continues to drop, will companies opt to build their own internal copilot service instead of using 3rd party tools or API services like GPT-4? Another interesting question is whether the proliferation of these models means that apps/services like GitHub will open up their copilot services to allow customers to "pick and choose" their own models to power their own copilot experience. I see some intriguing business model possibilities here... WDYT? #AI #LLM #Copilot

4

We have often seen many Customers asking one of the very interesting question. 🔥 We have deleted a large amount of data from some of the tables, but we still do not see the space reclaimed to disk. Why ? 🔥 🎇 Some of the database engines may also see performance degradation (no improvement) eventually, as the live tuples/records are scattered across more number of data blocks than actually required. This may be because the data that got deleted was scattered across a lot of data blocks but not really stored sequentially within a few data blocks. Queries on such tables may involve scanning more data blocks for a very few records. We call this as Table fragmentation. 🐘 This concern may not be limited to #postgres, the same is also applicable for #oracle or #sqlserver or any other database engine. 👨‍💼 👩‍💼 Database and Application Architects generally implement techniques like Partitioning and Sharding wherever applicable so that the challenges of data deletion/archival are completely avoided with a simple partition drop instead of a time and resource intensive "delete" statement. We will extend a little more on database scalability sometime later. For now, let us see what can we do to reclaim this space, if you are using PostgreSQL. 💰 Considering the resources and expertise available for #Oracle and #SQLServer like database engines, users might have already been aware of solutions to rebuild tables online in Oracle or SQL Server. 💸 Do note that such options are available only with Enterprise versions for most of the commercial database engines. For users migrating from #oracle to #PostgreSQL or newly deploying #postgres as the database engine, you could use one of the popular OpenSource extension called : pg_repack, that can safely rebuild tables online and reclaim the space back to disk. Manisankar K from HexaCluster has published this article on pg_repack and also compared with solutions available for the Oracle Enterprise versions. If you have a similar challenge where you could not avoid a massive delete but still want to rebuild the table online to reclaim space, read this article to safely implement pg_repack for PostgreSQL. Contact HexaCluster for expert assistance for your PostgreSQL databases and for database migrations to PostgreSQL. #postgres #postgresql #oracle #sqlserver #hexacluster #databasemigration #databasemaintenance

9

For GenAI applied to image, video and 3D, you need even more from the right cloud service provider: the latest GPU, the scalability for compute and storage at the right price, the right cost for ingress/egress data exchange (multimedia is an order of magnitude heavier than text).

16

1 Comment

NVIDIA NIM makes it easy for developers to build AI powered applications, but it also makes life easier for DevOps and IT teams who need to manage and scale these applications. Check out the new NIM Operator which facilitates the deployment, scaling, monitoring, and management of NVIDIA NIM microservices on Kubernetes clusters. https://lnkd.in/eEFEX_mT #generativeAI #Kubernetes #AIOps #MLOps

61

1 Comment

The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs by CeADAR Ireland CIADAR Ireland's new report dives deep into the art and science of fine-tuning Large Language Models (LLMs), bridging theory with real-world applications. Here’s a quick snapshot: ✨ Evolution of LLMs: From traditional NLP roots to driving AI innovation. 🛠 Fine-Tuning Techniques: Comparing supervised, unsupervised, and instruction-based methods for diverse tasks. 📈 7-Stage Fine-Tuning Pipeline: A structured roadmap covering everything from data prep to deployment. 💡 Efficiency Meets Performance: Spotlight on LoRA, Half Fine-Tuning, and advanced strategies like Mixture of Experts (MoE). 🤖 Aligning with Human Preferences: Exploring Proximal Policy Optimization (PPO) and Direct Preference Optimization (DPO). 🌐 Future Frontiers: Multimodal models, audio/speech fine-tuning, and tackling scalability and privacy challenges. Whether you're a researcher or practitioner, this report offers actionable strategies for navigating the evolving LLM landscape. Ready to dive in? Let’s innovate! 🚀

16

1 Comment

Great news from Microsoft-Quantinuum! They have achieved a great breakthrough in quantum computing by creating 4 logical qubits using 30 physical qubits, with no errors even after running measurements for 14,000 times. This is an impressive achievement, but it remains to be seen whether it can be scaled up for a large number of qubits, such as the 1000 qubits needed for large experiments. Check out the article for more details on this exciting development in quantum computing. #quantumcomputing #errorcorrection #qubits Source: https://lnkd.in/gxzvvZ5B

1

New Blog Alert: Maximize your fine-tuned model performance with the new Azure AI Evaluation SDK! In the journey of distilling large models like Meta Llama 3.1 405B into efficient smaller versions, evaluating the performance is crucial to ensure efficacy, accuracy, and readiness for deployment. 🔍 Learn how the Azure AI Evaluation SDK can: - Enhance your model assessment with advanced metrics - Provide insights with GPT-model based and math based evaluations - Enable bulk metric evaluation for comprehensive analysis Join me as we compare distilled models against baseline performance and explore valuable metrics that can drive continuous improvement. Whether you're in AI development or in search of efficient deployment strategies, this blog post is tailored for you. 👉 Read more: https://lnkd.in/gfQbNy3p #AzureAI #MachineLearning #AIEvaluation #AICommunity #ArtificialIntelligence #TechInnovation

30

Dealing with the dynamics of a digraph that looks like this.. or like that... New Generative models trained using not text but with real world graph examples. I expect this particular venture will lead to some unique and fascinating results.

2

Our episode on the The Data Exchange Podcast with our fantastic host Ben Lorica 罗瑞卡 , and featuring my great colleague Hima Patel , on data preparation for LLM applications (RAG, fine-tuning, instruct-tuning, and pre-training) using the Data Prep Kit is out! Data Prep Kit is the open-source (Apache 2.0 license) framework and transforms for language and code, which we used in IBM Research to prepare the data for the training of #IBM 's #Granite models ( https://lnkd.in/eUvZVK2t ). Running at any-scale, from a laptop to a datacenter and handling from a few pages of text to hundreds of billions of documents, it is available in pure-#Python , #Ray , #Spark and with #KubeFlow pipelines. Feel free to check it out here: https://lnkd.in/ehGjpbzG #IBMResearch #AI #GenAI #LLM

42

2 Comments

Check out this very insightful paper that explores the evolution of floating-point computation in scientific applications, highlighting how hardware and software have co-evolved to meet the growing demands of fields like artificial intelligence and high-performance computing. It also covers recent trends like mixed-precision computing and emulation, and the ongoing challenges of balancing precision, performance, and energy efficiency as computational workloads increase. This year’s Gordon Bell Prize winner for Climate, David Keyes and his team at KAUST, leveraged emulation techniques to reliever more efficient, high performance climate simulations.

11

Microsoft recently released GraphRAG as Opensource under the MIT license. Here is a brief summary of what it does. RAG, is retrieval augmented generation, which is simply scrape a source like the internet or a document for content related to the user's query and submit it to an LLM with the user's query so it can answer properly. GraphRAG, is the same thing, but you are using a knowledge graph instead. Graphs are nice because they have a bunch of relational patterns and algorithmic tricks to discover not readily understood connections. Here is how Microsoft is using GraphRAG. First, a graph is created of the data chunks you have made from a particular dataset. I couldn't find too much information on this part, possibly because they view graph creation as pretty standard, but I haven't seen yet if this package builds the graph as well or only interacts with it. That's the most important part in my opinion. Second, based on the maximum context window of whatever LLM you are using, cluster the graph into groups of related nodes and edges. The groups should be equal to totalTokenCountOfGraph / maxLlmContextWindow. Third, for each cluster, jam ALL of that cluster's data into the LLM with your user's query to get a report. Fourth, gather all of the clusters and keep answering until all of the answers are reduced to one final more relevant answer/summary. They report they are getting much better results than vanilla RAG, but in my opinion, this is pretty low brow and brute force. But that is pretty typical of the AI industry. I expect this to be rapidly improved though. There are a bunch of low hanging algorithmic improvements that can be made right off the bat. So, you can just keep an eye on it for now and see what kind of stuff knowledgeable graph experts or creative programmers start contributing.

6