Jeff Moser

I like Gradio (https://www.gradio.app/). It's better than Streamlit in a way that it supports building user interfaces right in the Jupyter Notebook environment. I built several exploration interfaces and it shows the potential for more advanced dashboard apps in Python. This is a univariate histogram explorer. More below 👇

30

2 Comments

"Prompt Engineering" by Lee Boonstra. This paper covers everything from basic prompting techniques to advanced strategies like Chain of Thought (CoT), Tree of Thoughts (ToT), and ReAct, along with practical tips for code-related prompting. Key sections include: - Configuring output with temperature, top-K, and top-P settings - Using zero-shot, one-shot, and few-shot prompting effectively - System, role, and contextual prompting techniques - Best practices, including experimenting with input formats and adapting prompts to model updates https://lnkd.in/gH5xrivh

13

This points to something I saw a few weeks ago at an AI conference here in Seattle. One presenter talked about neural net patterns that were much closer in structure to the way that neurons in our brain work, essentially reducing or even eliminating the back-propagation layer in favor of a trained stateful neuron system. It's both faster and requires far less energy to do many of the same the same things that one would use transformers for. Overall, this is what we need to be focusing on. It's becoming obvious that the transformer model architecture is simply not scalable - financial, ecologically, or even physically. What is beginning to emerge (and sadly getting far less funding) are some truly remarkable ways of building neural nets that don't require half the planet to run.

69

7 Comments

Something that only became obvious to me after I had been writing software for a couple of years is that there are different types of bugs, with different characteristics, impacts, and mitigation strategies. One class of bug that doesn't often get talked about is what I call the Semantic Bug - a Semantic Bug is basically an "oops I made some assumptions about this data/API/module that aren't really true" bug. For example: - "oops I thought this method was idempotent but it's not" - "oops I didn't think users could be active in two teams at once but they can" -"oops I didn't think these events could happen in that order but they did" I find it helpful to think of a dichotomy between logic bugs ("the code doesn't do what I intended") and Semantic Bugs ("the code does what I intended, but in the context of the larger system the thing I intended is wrong"). You might not run into Semantic Bugs too often when, say, working on a personal project, but they can be a real problem when developing larger software systems - especially when you have to reason about models and interfaces owned by other teams. Some types of bugs are fairly benign - if some unicode formatting looks weird in production, for example, you should fix it, but it's not going to break any functionality. It's generally ok to just squash these bugs when they show up and move on. You don't want to let that happen with Semantic Bugs, though, because they can be pretty nasty and unpredictable: -They don't always lead to a simple crash. In the worst cases, they can cause faulty components to exhibit essentially undefined behavior, potentially lying to other components about the state of the world in a way that’s hard to detect. Bad behavior can spread through the system, causing it to corrupt data or take inappropriate action. -They can be hard to catch with tests, if only for the reason that you’re apt to make the same faulty assumptions writing your test cases that you made writing your code. -They’re pernicious. Sometimes your wrong assumptions may only be wrong under very specific conditions, and you may be (un)lucky enough that it takes months for those conditions to occur in production. This can be maddening - when bugs occur directly after deploying a code change, you can at least point to a probable culprit, but when these “sleeper” bugs hit it feels like your happy, stable system keels over and dies for no reason. Semantic Bugs are really a product of our fallible human brains more than any technological constraint, so it might seem like there's not much we can do to prevent them besides making sure we're communicating well with our partner teams. The good news is that there are, in fact, architectural techniques you can use to make your software naturally resistant to Semantic Bugs! We put up a survey of existing approaches (along with a couple new ideas) at https://lnkd.in/g4mjqw2a - check it out if you're working on a large codebase!

14

pgai Vectorizer. Makes it possible to use one SQL command that will manage your vector embeddings "without" the usual engineering challenges to keep it in sync with your source data!

9

Models like OpenAI's CLIP combine text and images for search, but they have limitations, particularly with longer texts and complex relationships. This article talks about JINA's new model that improves on CLIP by integrating both text and image embeddings in one model, making it faster and more efficient for multimodal search tasks. Experiments and Findings: → Search with Both Text and Images: Combining both embeddings leads to better and more diverse search results, improving relevance and variety in results. → Image-Driven Diversification: By using images for diversification, Jina-CLIP v1 can present a broader range of results, offering users more varied options. Results Comparison: → Using both text and image embeddings (via averaging) consistently outperformed searches based solely on text or images, providing more precise and diverse retrieval across both broad and fine-grained categories. https://lnkd.in/guNRQpBU

15

Origins of Planning explores, how biological brains plans, interferes complex calculations and how machines can tackle hard problems in the cloud of probability. https://lnkd.in/gT2T2wS2

8

Hi all - in this post, I discuss self-supervision, a machine learning paradigm that sits somewhere between supervised and unsupervised learning. It is the reason we have many of the most powerful models available today, so I thought it would be fun and clarifying (for myself) to write a high-level summary of what it is and how it works. https://lnkd.in/eEVtWnCk Questions, comments, and corrections always welcome :)

38

3 Comments

🔮RAG + KG: Powerful Combination for Translation Multi-AI Agents Large language models (LLMs) have shown great promise in machine translation, but they still struggle with contextually dependent terms, such as new or domain-specific words. This leads to inconsistencies and errors that are difficult to address. Existing solutions often depend on manual identification of such terms, which is impractical given the complexity and evolving nature of language. While RetrievalAugmented Generation (RAG) could provide some assistance, its application to translation is limited by issues such as hallucinations from information overload. =================== In this paper, researchers propose CRAT, a novel multi-agent translation framework that: 》 leverages RAG and causality enhanced self-reflection to address these challenges. This framework consists of several specialized agents: ✱ the Unknown Terms Identification agent detects unknown terms within the context, ✱ the Knowledge Graph (KG) Constructor agent extracts relevant internal knowledge about these terms and retrieves bilingual information from external sources, ✱ the Causality enhanced Judge agent validates the accuracy of the information, ✱ Translator agent incorporates the refined information into the final output. =================== This automated process allows for more precise and consistent handling of key terms during translation. Their results show that CRAT significantly improves translation accuracy, particularly in handling context-sensitive terms and emerging vocabulary. --- If your company is looking for an AI consultant to accelerate business growth, or if you are looking to build your career in AI, send me a message and I can help

4

Happy to see that the community is still paying attention to encoder-based models 👏 TL;DR This blog post introduces ModernBERT, a family of state-of-the-art encoder-only models representing improvements over older generation encoders across the board, with a 8192 sequence length, better downstream performance and much faster processing. ModernBERT is available as a slot-in replacement for any BERT-like models, with both a base (139M params) and large (395M params) model size. https://lnkd.in/daH4aujE

15

Foundational models, or large language models (LLMs), train at the scale of the entire internet to acquire the vocabulary, syntax, and narrative structure of human language. Fine-tuning involves using a smaller, domain-specific dataset to grant additional specificity to a pre-existing foundational model that has already acquired patterns and features from massive underlying training datasets. By utilizing the existing knowledge embedded in a pre-trained language model, we unlock high-performance language understanding, but fine-tuning helps to get the right answer - not just an answer that sounds right - with a minimum of hallucinations or incorrect info. In healthcare, out-of-the-box, foundational models are often too generic to sufficiently answer the precision of a healthcare query. Foundational language models are trained on a massive scale of data, which includes jokes, incorrect information, misunderstandings, and misinformation, along with the sum total of human knowledge. The problem is, if you ask a large language model to give you factually correct information when it has been trained on BOTH accurate and inaccurate information - you’re playing a bit of a dangerous game. Enter fine-tuning language models. In the old days, we used to build a single model for a single task - for example, classifying procedures into evidence-based or not as laid out by clinical practice guidelines, for patients with particular disease markers. Each disease would have its own model, which took a long time to build and didn’t scale particularly well, and we struggled with the sparsity and inherent high-dimensionality of healthcare data. By fine-tuning LLMs, however, we're able to specialize those foundational models to achieve greater domain specificity and specialization, leverage the unique predictive and planning capabilities of LLMs, and cut down on the incorrect information that such models inevitably produce, while still preserving the underlying language prowess of the LLM. If improved accuracy and specificity wasn't enough of a selling point, there's another notable upside to fine-tuning foundational models for use in a clinical context. In terms of computational efficiency, using LLMs out of the box on specific tasks can require massive compute (and spend) that amounts to computational overkill for the problem. Some LLMs charge per query, some by compute load or time - either way, as a business running LLMs in a production environment, minimizing the number of LLM calls means optimizing your compute expenditures, too. When LLMs fail at scale in healthcare, it's usually along the fault line of failure to specify adequately. Fine-tuning large language models along domain-specific lines is one way to effectively productize genAI for healthcare use cases. #genAI #LLM #ai #healthcareAI #healthtech #largelanguagemodels #generativeAI #chatGPT

61

2 Comments

How do the costs compare of implementing a low-latency write-ahead-log (WAL) on S3 Express One Zone and one implemented as a State-Machine-Replication (SMR) system (such as Paxos/Raft/Kafka)? I built a cost model for both to find out. https://lnkd.in/dwczVutW

43

5 Comments

Valeriy Manokhin, PhD, MBA, CQF, I attended the Pydata conference back in 2016 and your recent post on the Dempster Shaffer method caught my attention. It brought to mind the valuable insights shared by Taposh Dutta Roy and Austin Powell from KP on this topic. If you're interested, you can find more details in this video: https://lnkd.in/gmegrbTb Love your posts Valeriy Manokhin, PhD, MBA, CQF keep them coming! #pydata

7

1 Comment

Very detailed slide deck from Mitko Vasilev talking about Open Source GenAI, setup, options, models, what and how it makes sense to do it yourself! Enjoy the good stuff yourself, I've been running models locally for some time now and it's really easy as one-click, so if you want to build a proof of concept and than scale it later into cloud, it's easy and very fun :) Great deck, Thanks Mitko! #genai #llm #opensource #diy

13

2 Comments

🌐 From the first compute optimized for spatial to the amazing Apache Sedona community, here are the reasons why I joined Wherobots. (post in comments) Last week I sent out a newsletter that covers these reasons in detail (if you aren't signed up make sure to do so) but here is the post that cover the reasons why I joined Wherobots. #gis #moderngis #spatialstack #geospatial #spatialanalytics #wherobots #apachesedona #spatialsql

82

2 Comments

Remember, AIs/LLMs built from training data are descriptive, not prescriptive. They are fine-tuned machinery to produce an output based on similar inputs and outputs it's been trained on. They predict what the output would have been had the input been present in its training data. They're descriptive. They are not applying novel intelligence to prescribe what the output should be. This is a key difference. Bias in the training data shines through in future output predictions. This is well-established. There's a predictive mechanism at play that is analogous to how Markov Chains work which you can hear me speak to briefly at this talk a few years back - https://lnkd.in/gAtynimB When LLMs are used for assisting software development this also means outdated patterns and practices shine through in its outputs. Technology moves incredibly quickly and it's actually very important that as of writing this ChatGPT 3.5's training data is 2.5 years out of date. Using 3.5 to develop software means you're ignoring several years of patterns and practices, and the code generation will demonstrate that. Two caveats: 1. GPT 4 is more recent as are some competitors 2. Custom GPTs allow for custom knowledge and instructions which may be more up to date #chatgpt #artificialintelligence #softwaredevelopment

19

Dealing with the dynamics of a digraph that looks like this.. or like that... New Generative models trained using not text but with real world graph examples. I expect this particular venture will lead to some unique and fascinating results.

2

This article delves into a novel approach to improving embedding models by increasing computational effort during inference (test time) rather than relying solely on scaling up pre-trained models. Think of it as "chain of thought" reasoning—applied to images! This innovative method provides a compelling alternative to scaling model size, offering a significant boost in performance without the associated costs of larger pre-trained models. Key Takeaways: - Scaling test-time compute can improve embedding models, much like how "chain of thought" reasoning enhances large language models. - By employing techniques like multi-step reasoning or attribute-based classification during inference, models can achieve higher accuracy without requiring larger pre-trained models. https://lnkd.in/gV6tQYyF

26