Drew Gray

Notion vs Glean reminds me a lot of Tesla vs Waymo... Just like I think Tesla is going to solve the 'software' problem of autonomous vehicles before Waywmo solves the 'hardware' problem... I think that Notion is going to solve 'enterprise knowledge search' before Glean solves the problem of document and knowledge creation... Why? A couple reasons: 1) Just like Tesla, Notion has a huge advantage when it comes to the amount of underlying data and documents to train on... They already have 100M+ users, and likely 10x that in terms of the number of documents, files, etc. in team's workspaces. 2) It's very hard to change people's behaviour re: where/how they create 'work'. It's why a lot of companies have come and gone in that category... As exciting as Glean's AI writing assistants might seem, most people are just fine (or even prefer) a good old fashioned editor/writer. I could be way off here, but this will be an intersting one to follow. Thoughts? Where am I getting it wrong? #ai #enterprise #knowledge #management #notion #glean

3

1 Comment

Wake up - new scaling laws paper just dropped! "Scaling Laws for Precision" Multiple interesting findings: * Training larger models in lower precision can be compute optimal - i.e. for a given compute budget you want to train larger models in lower precision rather than smaller models in higher precision in order to achieve the optimal loss * Too much pretraining data can actually be harmful as you quantize such models to lower precision for inference. Llama models popularized the data saturation regime (training on many more tokens / param than recommended by Chinchilla scaling laws (20 tok/param)) as their goal was not to train a compute optimal model but instead to optimize for inference. These models are more suceptible to degradations due to aggressive quantization. * When number of params is held fixed, compute optimal precision increases approximately logarithmically with data, i.e. for larger models you probably don't want to train in low precision even if your GPUs/training stack supports it! We're hitting the quantization limits that have historically brought us a lot of speed improvements (going from fp32 -> bf16 -> fp8). Lots more details in the paper: https://lnkd.in/dC3_EVrs --- 🔔 If you're looking to hire best ML/tech talent check out my platform here: https://lnkd.in/duJZ3D9T post a job here: https://lnkd.in/df-Ukm8E

169

4 Comments

We are now able to produce (just from text!) accurate 3D meshes through generative AI. There is still a long way to be totally production ready but the methods use can offer great compression. How about just in 2D? Checkout "An Object is Worth 64x64 Pixels: Generating 3D Object via Image Diffusion" from Simon Fraser University, City University of Hong Kong and CIFAR They introduce a new approach for generating realistic 3D models with UV maps through a representation termed "Object Images." This approach encapsulates surface geometry, appearance, and patch structures within a 64x64 pixel image, effectively converting complex 3D shapes into a more manageable 2D format. By doing so, they address the challenges of both geometric and semantic irregularity inherent in polygonal meshes. This method allows us to use image generation models, such as Diffusion Transformers, directly for 3D shape generation. Evaluated on the ABO dataset, our generated shapes with patch structures achieve point cloud FID comparable to recent 3D generative models, while naturally supporting PBR material generation. I really like the visuals in the video! More links in the comments #diffusionmodels #generativeai #meshgeneration #uvmapping #vfx #deeplearning #machinlearning #futurism

1,114

17 Comments

Benjamin Bolte from K-Scale Labs just posted his 'Robotics Predictions for 2025' containing some hot takes 🦾🔥 By the end of 2025.... 1. Humanoid hardware is commoditized - Hardware components will converge, but robot designs will remain different - Actuators no longer differentiators; industry moves towards open-source like MIT Cheetah - BOM for a humanoid will come down to a shocking <$8k! Question: What happens to incumbents like Telsa, Figure, 1X, Apptronik, etc., who have spent billions of $ in R&D to create robots with a $50 - $150k BOM? 2. No ChatGPT moment for robotics - A LOT of data (millions of hours) is needed to train a general-purpose robotics model - Language + multi-modal models will not be able to control a robot sufficiently - Progress will be incremental rather than spiky Question: If building a robot for a domain and bootstrapping to generality is a losing strategy, what's a better one? 3. More than half of humanoids will be sold to consumers - The 'killer apps' will be speech and vision - Traditional SLAs for business customers will force robotics teams away from building good generalist models - "Robotics companies will start looking more like SV tech companies, caring about user engagement and retention." Question: What are the 'killer' consumer use cases that will drive this adoption? What are the jobs to be done? Link to the full blog in the comments. What are your thoughts?

26

2 Comments

ImageRAG Powered by Llama 3.2 Vision ImageRAG is a powerful RAG system combining Colpali's ColQwen image embeddings with Ollama LLaMA 3.2 Vision. 🌟 Key Features of ImageRAG 🧬 ColQwen model for generating powerful image embeddings via Colpali 🤖 LLaMA Vision integration through Ollama for image understanding 📥 Intelligent image indexing with duplicate detection 💬 Natural language image queries 📄 PDF document support 🔍 Semantic similarity search 📊 Efficient SQLite storage 🛠️ Technical Stack of ImageRAG Embedding Model: ColQwen via Colpali Vision Model: LLaMA Vision via Ollama Frontend: Streamlit Database: SQLite Image Processing: Pillow, pdf2image ML Framework: PyTorch #imagerag #rag #multimodalrag #nlproc #llms

818

34 Comments

ImageRAG Powered by Llama 3.2 Vision ImageRAG is a powerful RAG system combining Colpali's ColQwen image embeddings with Ollama LLaMA 3.2 Vision. Key Features of ImageRAG 🧬 ColQwen model for generating powerful image embeddings via Colpali 🤖 LLaMA Vision integration through Ollama for image understanding 📥 Intelligent image indexing with duplicate detection 💬 Natural language image queries 📄 PDF document support 🔍 Semantic similarity search 📊 Efficient SQLite storage Technical Stack of ImageRAG - Embedding Model: ColQwen via Colpali - Vision Model: LLaMA Vision via Ollama - Frontend: Streamlit - Database: SQLite - Image Processing: Pillow, pdf2image - ML Framework: PyTorch **ImageRAG details in the comments #imagerag #rag #multimodalrag #nlproc #llms

912

46 Comments

🌭🌭HOP: Hand-Object interaction Pretraining🌭🌭 👉Berkeley unveils a novel approach to learn general robot manipulation priors from 3D hand-object interaction trajectories. Using in-the-wild videos to generate sensorimotor robot trajectories. 𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬: ✅General manipulation prior from in-the-wild ✅Simulator as videos-sensorimotor intermediary ✅Fine-tuning via RL and behavior cloning (BC) ✅Sample-efficient adaptation to downstream tasks #artificialintelligence #machinelearning #ml #AI #deeplearning #computervision #AIwithPapers #metaverse 👉Discussion https://lnkd.in/dMgakzWm 👉Paper https://lnkd.in/dhRFQara 👉Project https://lnkd.in/dCa77u7P

168

3 Comments

The market demand for robotic packing of items is ever-increasing in the intralogistics and e-commerce order fulfilment sectors. We are prepared to serve our customers in their material handling use-cases

8

Struggling with noisy RMSD plots? 🧪 Here's how to reduce noise and gain actual insights from protein structure comparisons through RMSD: 1️⃣ Superimpose the structures to eliminate noise caused by orientation differences. 2️⃣ Exclude side-chain atoms to focus on meaningful secondary structural changes. Most tools like Chimera and GROMACS calculate RMSD after superimposition by default. However, they allow you to choose the atom set—Backbone, C-alpha, or the whole protein. The Backbone RMSD is ideal in most cases, offering clarity and reliability. In future posts, I’ll discuss when and why to narrow down to C-alpha RMSD for specialized analyses. Feel free to reach out for discussions on Molecular Docking, MD Simulations, and more! Let’s dive into science together! 🚀✨ Follow me for more such useful Structural Biology and Bioinformatics posts!!! #Proteins #RMSD #MolecularDynamics #StructuralBiology #Bioinformatics #ComputationalBiology #ProteinDynamics #ResearchSimplified #ScienceCommunication #DataAnalysis 

54

2 Comments

Berkeley team have developed an enhanced method, called LoRA+, to efficiently fine-tune large language models using Low Rank Adaptation (LoRA). LoRA reduces the number of trainable parameters by introducing low-rank matrices A and B to adapt pretrained weights. However, the standard LoRA setup, which applies the same learning rate to both A and B, leads to suboptimal feature learning for very wide models.     The authors show mathematically that the optimal learning rates for A and B should increase differently with model width. LoRA+ addresses this by using different learning rates for the two matrices, with the learning rate of matrix B being a fixed multiple λ of the learning rate of matrix A. Guidelines provide recommendations for choosing λ based on model architecture and task.   Experiments validated that LoRA+ improves both fine-tuning speed and performance compared to standard LoRA for GPT-2, RoBERTa, and LLaMA models on various benchmarks. Benefits were greater for tasks requiring efficient feature learning to align the model. Integrating LoRA+ is straightforward, requiring only setting the learning rate ratio hyperparameter λ, with a recommended default of λ=16.    This work provides guidance on hyperparameter settings for LoRA, a standard for efficiently specializing large language models. By showing how to "scale up" LoRA for today's massive models, LoRA+ is a useful tool for practitioners wanting to leverage powerful foundation models while controlling costs.    Arxiv: https://lnkd.in/ep6MHyWh

1

Human On the Loop Demand Aware Robot Scheduling: A Mixed Reality Based User Study, IEEE International Conference on Robot and Human Interactive Communication (Ro-MAN) 2024 #robotics #mixedreality #multiarmbandit #reinforcementlearning https://lnkd.in/gM7mchke Ajay Kumar Sandula Rajatsurya Madhusudhana Debasish Ghose Scheduling tasks and subtasks for multiple heterogeneous robots poses a significant challenge, particularly in scenarios where human supervision is essential. The complexity is further compounded when considering human factors in the scheduling process, especially in demand-aware task scheduling environments (tasks are generated based on external demand). The primary goal of this research is to understand human decision-making and how it affects their scheduling process, and we undertook two studies. In the first study, we employed a mixed reality based user study to explore how human perception of the scheduling environment influences task scheduling and facilitates personalized resource allocation. Our findings indicate that human task schedulers exhibit enhanced performance when assisted by autonomous agents, compared to scenarios with limited autonomy in robotic systems. To explore the impact of robot planning on human decision-making and its effects on task scheduling and collision-aware scheduling capabilities, We conducted the second study. This study employed a mixed reality-based warehouse environment, where two users controlled different robots with shared objectives. The findings suggest that human operators exhibited improved collision-aware scheduling without compromising their demand-aware scheduling capabilities when visual aids such as collision cones were incorporated.

22

Great idea to generate 3D images from 2D images.

2

Great explanation and visualization, thanks for sharing Alex Razvant! I would add that other graph compilers have a similar flow, but the differentiation is in the implementation details. For modern AI architectures, graph compiler optimizations are not enough to achieve high performance and utilization, and many AI compilers have dedicated implementations. FlashAttention (kernel-fusion) and PagedAttention (memory management) are examples of dedicated optimization techniques that are not achieved by generic compilation (not to mention Speculative decoding and other algorithmic "tricks"). By the way, this is the main reason in my opinion for why we see so many competing inference engines for LLMs (TensorRT, vLLM, TGI, LMDeploy, ONNX Runtime, SGLang, CTranslate2, llama.cpp, MLC-LLM, DeepSpeed, IPEX-LLM, etc.) 😅

19

1 Comment