Ethan Kim

Ilya Sutskever confirmed at NeurIPS '24 pre-training scaling law will not be the same. Internet data, the AI fossil fuel, is running out. It is not the end of the AI hype, the rapid rise of foundational AI has outpaced our ability to fully grasp their long-term societal impact. AI agents and applications are poised to transform every facet of our lives, and we will see it happening in coming few years. There will still be improvements in model architecture, and synthetic data creation, so models will improve, just not at the lightning speed they did till now. Looking forward, the single most important milestone in AI is creating systems capable of advancing themselves—AI researchers that can innovate and improve foundational models autonomously. We need to assess how far are we from this stage.

9

While you probably don't have to worry about aliens invading on the fourth of July, you should be aware that text embeddings may expose your private information. Fortunately it's an addressable risk. Our Senior AI Scientist, Joseph Ferrara, PhD, covers how to mitigate the problem on a pdf. Broadly, the steps are: -Extract the text -De-identify the sensitive info -Chunk the resulting text -Embed it in a Pinecone database -Perform some test queries If you want to see more detail and sample code, check out the how to guide the comments. #RAG #AI #embeddings

15

3 Comments

🚀 Unlocking the Power of Variational Autoencoders in Time Series Analysis! 📊 I'm excited to share my latest Substack post, where I dive deep into the fascinating world of Variational Autoencoders (VAEs) and their application in generating synthetic time series data. 🔍 What You'll Discover: A clear explanation of how VAEs work, including the encoder and decoder architecture. Step-by-step code to implement a VAE for time series data generation, complete with visualizations of the results! This post is perfect for data scientists, machine learning enthusiasts, or anyone curious about generative models and their practical applications. Let’s unlock new insights in data generation together! 🌟 #MachineLearning #DataScience #VariationalAutoencoder #TimeSeries #AI #DeepLearning #DataGeneration

2

🎯Data was, is, and will be the only fuel. Analogy with the fossil fuel is a good one, since the data we have today is as dirty as oil and coal. What big tech companies tried is to put a lot of "garbage" data, and what they did get was fantastic. Yet, we are at the point of diminishing returns. Data is the beginning and the end of this cycle.” Great thread, particularly this comment 👆from Ilija Lazarevic

20

2 Comments

I have to write outreach emails/messages regularly to potential clients. I wanted to automate the writing process. I could use LLMs to write it for me, but I hate it when it sounds more like a robot and it's even worse when it hallucinates 😞 So I created a RAG-based LLM application (running locally) using LlamaIndex that takes in all my previous emails, proposals, and portfolio projects and generates content that sounds more like me 😎 . To top it off, it often identifies and highlights connections between our previous work and the client's industry (See the demo below for a closer look.) Note: This is just a demo, I haven't provided it with much data yet. I think it can do even better with more personalized content. #LLM #RAGtechnique #AI #outreach #codeclever

26

7 Comments

ok - given twitter/x is the place where bleeding edge R&D happens (if you can evade the AI hype and crypto posts successfully), I'm trying to figure out how much cross-over there is over here in what i think is more of the real world. 1. have you heard of the 'anon' entropix project? (uses parallel sampling, with a draft model, and a bunch of other inference-level 'hacks' to the tokenizer and decoder) 2. have you tested it? (i did for ~1 hour yesterday, not sure what to think yet) 3. has anyone seen anything neat with speculative decoding + draft models (haven't seen a lot in the open since medusa)? one thing i know i like about entropix is that it uses a base model (without chat/instruct tuning) + an instruct in parallel. base models are powerful - even 1B parameter ones. tldr: if there's any activity/comments on this post, i will be shocked, but genuinely curious to get the real world's perspective. if you can get really fantastic results by running two 1B parameter LLMs on certain tasks, that clears the way for running almost everything locally, on-device, privately, and only offloading to cloud/api services what you can't do locally.

12

7 Comments

I’m honored to have my insights on synthetic data validation for Large Language Models (LLMs) featured in Label Your Data’s latest article from Featured. Now that many companies build specific models (e.g., Llama 3.1 405B) to facilitate synthesis dataset generation, integrating human expertise into the LLM development process is important. Human-in-the-loop (HITL) approaches ensure that domain experts guide and refine model outputs, leading to more accurate and contextually relevant results. As a GenAI researcher, I believe the concern of running of training data for future LLMs development in the future, raised by experts in the field like Chip Huyen, is valid and deserve significant awareness. The expert-informed data synthesis validation helps mitigate biases and enhances the model’s ability to handle complex, nuanced scenarios, for example thinking about if a biotech company needs to generate synthesis customer reviews for a niche healthcare product not lanuched yet or a forensic psychiatry research group aims to generate crime vignettes to assess the public’s perception of crime types and punishment relevant to demographic groups. Thank you again to the Label Your Data team for highlighting this important aspect of AI development. #AI #GenAI #SyntheticData #LLM #DataValidation #HITL #ResponsibleAI

24

If you think compute scaling will plateau, think again. We are in the baby states of inference time compute. VLLMs coupled with computer tools, reward learning and RL can leverage orders of magnitude more inference time compute than what we have seen with foundation models. Here’s a concrete training architecture (replace \pi with a large pre-trained vLLM and add it to the reward learner too, \R is the space of all computer tools)

22

1 Comment

Haven't posted much recently - but wanted to share about three papers I've been deep into studying lately: auto-encoders for time series clustering, utility and enjoyment of video games over time, and matrix factorization for gene selection. I love how these seemingly unrelated concepts can come together. While all three papers have unique problem statements and methods, I've found its really helped to consider such different ideas in tandem. Since studying I've had a few succesful ideas combining learnings from each paper, coupled with even more confusion and ideas that do not work (like, at all). But, I find that has been my journey of learning mathematics and I'm trying not to be so hard on myself when I don't understanding things on the 1st, 2nd, or even N'th try. That being said, being able to combine methods used for gene selection with that of video game utility however has been a pretty great payoff! Links to the papers below.

28

9 Comments

📜 A new blog post explaining the fundamentals of RL for LLMs—the perfect place to intuitively understand PPO and RLHF from first principles. 👇👇 Pre-trained LLMs are often compared to Lovecraft's Shoggoth, a monstrous, shape-shifting beast. PPO is the algorithm that brought it under control. Take a look! 🔗 https://lnkd.in/eXqDvrUV

29

1 Comment

🚨 BREAKING NEWS! 🚨 🔥 Latttice + Claude.ai = A NEW ERA in Data Exploration! 🔥 🚀 Get ready to supercharge your data with the power of AI! With Claude.ai integrated into Latttice, accessing and exploring data has never been this EASY or EXCITING! 💥 💬 Talk to your data! Ask questions in plain English and get instant, actionable insights. No more complicated queries—just results! 🔗 Seamless integration with all your data, NO coding required. It’s as simple as having a conversation. 🤖 🔒 Security that’s enterprise-grade—because your data deserves the best protection. 🛡️ 💡 Say goodbye to guesswork and hello to data-driven brilliance! Ready to take your decision-making to the next level? 🌟 👉 Join our Beta Program TODAY and experience the future of data firsthand! #AIRevolution #DataExploration #ClaudeAI #Latttice #NoCode #SmarterDecisions #TechInnovation #FutureOfData #AI #DataScience #BetaTest

3

1 Comment

If you've ever worked with ML models for medical imaging then you know how long the computation time will be. Here's a quick tip help with that. If you deal with CT or MRI scans of hundreds of slices then your model will take minutes to finish. Now imagine having multiple of these models working together for a certain application! Inference time can quickly get out of hands! In such a case, you most likely have a model orchestrator which is a piece of code that sends requests to your models APIs and then collect the outputs and order them. What you should do is leverage asynchronous processing. Meaning, run all of your models in parallel at the same time without waiting for one model to finish before calling the second one. Of course, this supposes that each model is deployed separately in its own API. But what if some models depend on other models outputs? You can still use asynchronous processing, but you'll be picking and choosing which models need to run first and which models need to wait a little bit. A great package to do this in Python is asyncio.

32

Our team at HRai Labs is incredibly excited to announce the release of the Employee Equity Toolkit, a free, easy to use, and fully open-source disparate-impact-analysis tool for your business: https://app.hrailabs.com/. In an era where diversity, equity, and inclusion (DEI) are at the forefront of organizational values and strategy, having the right tools to monitor and analyze workforce data is essential. It shouldn’t cost hundreds of thousands of dollars a year to outsource disparate impact studies to a legal firm. These studies aren’t dynamic, don’t allow for more real time analysis, and are one-time engagements. Most importantly, employees should never face disparate impact due to organizations not regularly conducting these critical studies. That’s why HRai Labs created this open-source toolkit for organizations to run these analyses: https://lnkd.in/g3muuvgJ. We invite you to explore the Employee Equity Toolkit and leverage data to promote fairness and equity in your workplace. Ready to learn more? Check out our Medium article explaining the details or visit us at hrailabs.com.

5

Looks like I've been quite lucky today - Found a 3rd article which seemed really valuable and worth sharing How can we be sure the LLM isn't generating something incorrect or gibberish? Can't just give an unevaluated output to our user, now can we! Santiago Valdarrama has mentioned a few resources that can help set up an evaluation pipeline to assess the quality of the LLMs outputs. #goodread #datascience

2

For too long the world has blindly accepted that AI models are black boxes but recent advances in the science of AI interpretability have demonstrated that these systems have a profoundly rich internal structure. While larger and larger AI systems are actively being trained on vast amounts of data, reliability and safety remain huge challenges that are exacerbated by our collective lack of insight into what these models have actually learned. That's why my cofounders Tom McGrath, Eric Ho, and I decided to found Goodfire. At Goodfire we are building the foundational technology which will make interpretability the default. We envision a world where nobody deploys AI models that they don't themselves understand thereby making AI systems something we can truly design in the service of humanity. We are looking for exceptional engineers and scientists to join our founding team. If you deeply understand the importance of our mission and want to help build a future where humanity is empowered to architect AI systems in safer and more robust ways please reach out! Read about us here Website: https://goodfire.ai/ Press release: https://lnkd.in/g_SBCU4Z VentureBeat coverage: https://lnkd.in/gU3xvPp9 Expressing deep appreciation for our investors at Lightspeed, Menlo Ventures, Work-Bench, Juniper Ventures, South Park Commons, Mythos Ventures, Bluebirds Capital, and our numerous angel investors for believing in us and our vision, and to the amazing scientific community who has been working hard for many years to make this foundational technology at last possible.

111

45 Comments

🚀 [Armand's Software Projects 06: Word.NLP] 🚀 In today's post, in my short tenure with Backdrop Labs, I've focused on a simple Natural language processing app that utilizes python, streamlit, pandas, NLTK, etc, with the objective to auto correct and auto-suggest certain words. ⚡How it Works!⚡ This app uses a pre-loaded notebook.txt file containing common words, which have been cleaned and converted into utf-8 encoding. This notebook also contains their word frequencies. The app will analyze a user's input, and using NLTK, will calculate the edit distance between the inputted word and every word in the vocabulary dataset. The suggestions are further ranked based on their frequency in the vocabulary. After that, the streamlit app will display any correct or in-correct words and their probabilities. A few downsides to this project is: 1. A bit too simple: This is a very simple streamlit project that can be improved upon later on. Maybe even TensorFlow's tokenizer or other NLTK methods and functions can help develop this further on. 2. Limited dataset: The dataset that was used is limited and doesn't contain all of the words in the English Language. 🔗The link for this Streamlit app and Github repository is here! 🔗 https://lnkd.in/g-vk4ync https://lnkd.in/gX78zmzx #NLP #Python #nltk #streamlit

31

5 Comments

Increasingly AI applications invoke tools, leverage resources/data and have multiple steps and models involved in a complex pipeline. The boundaries between "RAG", "agents" and other names we have given these compound AI systems are blurring. For example, one thing that would be super useful is hyperparameter-style optimization for RAG pipelines. You define the knobs of your system, and the search space, and we run many sweeps to find the best configuration. Some knobs could be document chunking, embedding model or retrieval-time LLM used. But other knobs could be whole algorithms or techniques, such as contextual retrieval (https://lnkd.in/ejH9vCXZ), graph RAG, etc. What's clear is there are so many permutations for increasingly complex AI systems that we need tools to help systematically measure what works and doesn't work for *your* application. Evals is just the starting point -- to take those learnings and improve your applications with it requires methodical experimentation.

23

4 Comments

🔍 Ever wondered how deep learning models "see" complex patterns? In my latest Medium post, I dive into GoogLeNet, the groundbreaking architecture that redefined how neural networks capture details. From multi-branch Inception blocks to its unique layered design, this model goes deeper than ever to enhance feature extraction. Curious how GoogLeNet powers up deep learning? Read on to see how I applied it to the Fashion-MNIST dataset! https://lnkd.in/dwrhv4wZ #DeepLearning #Datascience #AI #NeuralNetworks #MachineLearning

7