Karol Hausman

While a small number of mathematicians have since accepted that Mochizuki's papers prove the conjecture, other researchers say there are holes in his argument and it needs further work, dividing the mathematical community in two and prompting a prize of up to $1 million for a resolution to the quandary. https://lnkd.in/e9xfpHJz #InnovateForImpact #technology #tech #innovation #engineering #science #software #huawei #design #electronics #fourthindustrialrevolution #programming #coding #education #computer #mathematics #artificialintelligence #math #uni #student #college #phd #research #quantumcomputing #quantummechanics #physics #technology #quantumcomputer #industry40 #artificialintelligence #ai #machinelearning #emergingtechnologies #computerscience #maths #innovation #settheory #it #quantumtechnology #datascience #mit #sustainability #it #hydrogen #computerscience #logic #theoreticalphysics #data #finance #asic #infrastructure #manufacturing #digitaltransformation #futureofwork #datascience #teamwork

1

Great explanation and visualization, thanks for sharing Alex Razvant! I would add that other graph compilers have a similar flow, but the differentiation is in the implementation details. For modern AI architectures, graph compiler optimizations are not enough to achieve high performance and utilization, and many AI compilers have dedicated implementations. FlashAttention (kernel-fusion) and PagedAttention (memory management) are examples of dedicated optimization techniques that are not achieved by generic compilation (not to mention Speculative decoding and other algorithmic "tricks"). By the way, this is the main reason in my opinion for why we see so many competing inference engines for LLMs (TensorRT, vLLM, TGI, LMDeploy, ONNX Runtime, SGLang, CTranslate2, llama.cpp, MLC-LLM, DeepSpeed, IPEX-LLM, etc.) 😅

19

1 Comment

Mr. Jensen Huang at NVIDIA DOES know about the importance of culture! One week ago I posted a comment regarding an article I read about NVIDIA's culture. Back then I stated that I do not know if the article was accurate as I am not an employee. Today I am providing an update. I read this new article yesterday and I expanded my view of NVIDIA and Mr. Jensen Huang and had to update my comment. While reading the attached, I found myself thinking, 'What is this? A CEO who actually knows what they're doing when it comes to creating a purposeful corporate culture?" Yes, that is correct. I want to call it out publicly that if the article attached is accurate, Mr. Huang IS building a purposeful culture and I want to say "THANK YOU"! It's rare to find executives who invest the time in building a purposeful PERFORMANCE Enhancing Corporate Culture. Yet there is one section labeled "Improved Loyalty" that needs addressing. Specifically the following "As Millennials and Gen Z become the dominant segments in the labor market, they prioritize work-life integration over strict work-life separation where salary is the focal point. They also value mentorship and coaching more than previous generations, such as Gen X and Baby Boomers." That statement is 100% incorrect. In the early 1980s (maybe late 1970s) most companies decided to terminate the social contract that built employee loyalty. They terminated advanced and executive education, pensions, mentorships, on-site healthcare (for larger companies), and other benefits beyond salary. Employees came to realize that they were only getting a salary with partially compensated benefits that were easily comparable to other offers from other companies. That does NOT mean that mentorship and coaching were less valued by Gen X or Boomers. It means it was less available, if at all. I know this for a fact because I worked at Texaco Inc., a Fortune 25 at the time. I was fortunate to have been assigned different mentors, coaches, and education until I saw that disappear in 1992. Texaco's HQ was between IBM and PepsiCo and I interacted with both of those companies and their employees regularly, and I saw it there as well. This "TERMINATION" of the employee / employer loyalty contract was NOT the decision of the employees' due to lack of value. It was seen as an unnecessary perceived cost. Mr. Huang has it right. Treat people like you want them there and demand that they grow, develop, and continually add more value as members of the team while showing them respect and holding them accountable, and they will be loyal. Treat them like they are easily replaced and they will leave for more money, more responsibility, and a more demanding role. I am glad I am a shareholder as I am impressed by Mr. Huang's open discussion of his approach. https://lnkd.in/emz6mMAK #culture

3

2 Comments

Effortless Integration for Smarter Insights: CloudEagle.ai Meets Bitbucket Integrating CloudEagle.ai with Bitbucket takes just minutes and gives you: ✔ Simplified App Tracking: No more manual reports, get the app usage data you need, when you need it. ✔ Centralized Control: Manage your Bitbucket security from the CloudEagle dashboard. ✔ Proactive Monitoring: Catch unusual login activity before it becomes a problem. Ready for a smarter, more secure development workflow? Book a demo today - https://lnkd.in/gXAhChrt #saas #saasmanagement #toolstuesday Atlassian #jira #toolstuesday

5

Excited to share our lab's latest research in 3D image/video compression by Stephen Siemonsma: "N-DEPTH: Neural Depth Encoding for Compression-Resilient 3D Video Streaming" 🥪 To our knowledge, no existing work has directly optimized the end-to-end depth reconstruction losses by sandwiching deep learning networks around a lossy image compression codec. ✨ We present N-DEPTH, a compression-resilient neural depth encoding method that leverages deep learning to efficiently encode depth maps into 24-bit RGB representations that minimize end-to-end depth reconstruction errors when compressed with JPEG. N-DEPTH’s learned robustness to lossy compression extends to video codecs as well. Compared to an existing state-of-the-art encoding method, N-DEPTH achieves smaller file sizes and lower errors across a large range of compression qualities, in both lossy image (JPEG) and video (H.264) formats. 💡 This work has immediate implications for 3D streaming, telepresence, and other emerging applications requiring high-quality 3D depth data transmission. 📄 N-DEPTH was published in a special issue of Electronics and is available via open access here: https://lnkd.in/e-nfFvme #3D #DeepLearning #AI #ML #Compression

65

1 Comment

Human On the Loop Demand Aware Robot Scheduling: A Mixed Reality Based User Study, IEEE International Conference on Robot and Human Interactive Communication (Ro-MAN) 2024 #robotics #mixedreality #multiarmbandit #reinforcementlearning https://lnkd.in/gM7mchke Ajay Kumar Sandula Rajatsurya Madhusudhana Debasish Ghose Scheduling tasks and subtasks for multiple heterogeneous robots poses a significant challenge, particularly in scenarios where human supervision is essential. The complexity is further compounded when considering human factors in the scheduling process, especially in demand-aware task scheduling environments (tasks are generated based on external demand). The primary goal of this research is to understand human decision-making and how it affects their scheduling process, and we undertook two studies. In the first study, we employed a mixed reality based user study to explore how human perception of the scheduling environment influences task scheduling and facilitates personalized resource allocation. Our findings indicate that human task schedulers exhibit enhanced performance when assisted by autonomous agents, compared to scenarios with limited autonomy in robotic systems. To explore the impact of robot planning on human decision-making and its effects on task scheduling and collision-aware scheduling capabilities, We conducted the second study. This study employed a mixed reality-based warehouse environment, where two users controlled different robots with shared objectives. The findings suggest that human operators exhibited improved collision-aware scheduling without compromising their demand-aware scheduling capabilities when visual aids such as collision cones were incorporated.

22

NVIDIA's best GPUs are anywhere from 3x as good to 1.3x as good as AMD's for certain kinds of tasks, where "good" is defined by amount of computing per dollar (and, although not shown here, per watt). I could understand AMD not being able to win some business, or even being able to charge a premium for its products beyond their proportionate performance gains. But NVDA is taking nearly ALL of the AI business. And AMD's market cap is... 6% of NVDA's. Why? It isn't worth it, at some price, to use AMD chips? The answer is software. The reality is that it's crazy hard to even benchmark most of the AI workloads on AMD stuff because of the inadequate software ecosystem. The catalyst for the breakout of AMDs stock is... AMD making progress on narrowing the software gap OR a fundamental paradigm change in the software tooling required for both. https://lnkd.in/d6ucgQ2E

20

1 Comment

📣 Recurrent Drafting (ReDrafter): a novel #LLM speculative decoding technique, open-sourced by Apple, is now on NVIDIA TensorRT-LLM, improving performance of LLM inference by 2.7x. Learn how to use ReDrafter to boost LLM performance.

1

https://lnkd.in/eXbCsAnD “He who controls the GPUs, controls the universe.” Access to GPUs, and in particular those made by Nvidia, the leading supplier, is vital for any company that wants to be taken seriously in #ai. Nvidia’s dominance has driven its market value to more than $2trn. 'GPUs do the computational heavy-lifting needed to train & operate large #ai models. Yet, oddly, this is not what they were designed for. The initials stand for “graphics processing unit”, because such chips were originally designed to process video-game graphics. It turned out that, fortunately for Nvidia, they could be repurposed for #ai workloads. 'Might it be better to design specialist ai chips from scratch? That is what many companies, small & large, are now doing in a bid to topple Nvidia. Dedicated ai chips promise to make building and running ai models faster, cheaper or both. Any firm that can mount a credible threat will have no shortage of customers, who dislike its lofty prices and limited supplies. 'GPUs contain several thousand processing engines, or “cores”, which let them run thousands of versions of the same simple task at the same time. Running #ai models similarly involves running lots of copies of the same task in parallel. Figuring out how to rewrite ai code to run on GPUs was one of the factors that triggered the current ai boom. 'Yet GPUs have their limitations, particularly when it comes to the speed with which data can be shuffled on and off them... When training very large #ai models, some GPu cores may be idle as much as half of the time. Andrew Feldman, the boss of Cerebras, a startup based in Sunnyvale, CA, likens it to the gridlock in a grocery store on the day before Thanksgiving. “Everybody’s in a queue, so there are blockages in the parking lot, there are blockages in the aisles, blockages at the checkout. That’s exactly what’s happening with a gpu.” 'Cerebras’s response is to put 900,000 cores, plus lots of memory, onto a single, enormous chip, to reduce the complexity of connecting up multiple chips & piping data between them. 'Groq, a startup from Mountain View, CA, is taking a different approach. Its ai chips, called language processing units (lpus), are optimised to run #llms particularly quickly. 'Yet another approach is that taken by MatX, also based in CA. GPUs contain features & circuitry that provide flexibility for graphics, but are not needed for llms. The firm is working on getting rid of such unnecessary cruft.

14

3 Comments

I have mad respect for Nvidia and all who work there. But as the company that more or less rose from the ashes of Silicon Graphics, it isn't potentially making the same core mistake? Its chips get ever more powerful AND power hungry, akin to building massively centralized supercomputers. We recall the thing that killed SGI was missing the move to PCs and effective decentralization and commodification. Today that means "mobile"--as in low-power on-device AI/neural processing built into custom silicon by companies like Apple and lately Microsoft.

25

4 Comments

Berkeley team have developed an enhanced method, called LoRA+, to efficiently fine-tune large language models using Low Rank Adaptation (LoRA). LoRA reduces the number of trainable parameters by introducing low-rank matrices A and B to adapt pretrained weights. However, the standard LoRA setup, which applies the same learning rate to both A and B, leads to suboptimal feature learning for very wide models.     The authors show mathematically that the optimal learning rates for A and B should increase differently with model width. LoRA+ addresses this by using different learning rates for the two matrices, with the learning rate of matrix B being a fixed multiple λ of the learning rate of matrix A. Guidelines provide recommendations for choosing λ based on model architecture and task.   Experiments validated that LoRA+ improves both fine-tuning speed and performance compared to standard LoRA for GPT-2, RoBERTa, and LLaMA models on various benchmarks. Benefits were greater for tasks requiring efficient feature learning to align the model. Integrating LoRA+ is straightforward, requiring only setting the learning rate ratio hyperparameter λ, with a recommended default of λ=16.    This work provides guidance on hyperparameter settings for LoRA, a standard for efficiently specializing large language models. By showing how to "scale up" LoRA for today's massive models, LoRA+ is a useful tool for practitioners wanting to leverage powerful foundation models while controlling costs.    Arxiv: https://lnkd.in/ep6MHyWh

1

Smaller #genAI models are changing the game! Introducing #mistral-NeMo by #nvidia and #mistralAI with 128k token context window - a tool for businesses that can run on RTX GPUs which you may have at your home. Super inexpensive with no high cloud costs, keeps data private on your private computer! #nvidia #mistralai #genaiforenterprise #responsibleai

10

🤖 Quantinuum H2 improvements say a lot about quantum advantage. This is a bit of a different approach to evaluation than previously (though if you want a brief breakdown on the technical implementation, check out today’s newsletter). But, this paper from Quantinuum is significant in its ties to the bigger picture. TLDR: Improvements to Quantinuum's H2 quantum processor allow it to operate with up to 56 qubits and high connectivity while maintaining high two-qubit gate fidelity, as demonstrated with random circuit sampling tasks which are known to be challenging for classical computers. Overall, the improvements to H2 widen the gap between classical ability and quantum ability. As a writer and reader within the space, I can tell you that it’s easy to get lost in the increasingly dense zoos of algorithms, error correction codes, minutiae of various updates to modality architectures, etc. As such, I’ve been considering personal benchmarks for evaluating the value of new papers they can be briefly summed up as follows: ⚆ We need to get answers that weren't accessible before (this can include getting "better" answers), and these answers must be tied to practical applications. ⚆ Energy efficiency must be considered -- if we get a better answer at a significantly higher energy cost, was this truly better than the classically “good” answer? (aka practicality in implementation) ⚆ And, the benefit of applying the solution to the practical application must outweigh the energy cost of achieving that solution. Keeping this in mind, Quantinuum set about to improve their existing hardware in terms of connectivity and fidelity. An aside on connectivity because it doesn't always get the recognition it deserves -- connectivity refers to the capabilities of the hardware in terms of the arrangement of qubits and the available gate operations. Higher connectivity means more direct implementation of quantum gates between a pair of qubits. This helps entanglement as it reduces the number of intermediate operations needed and as such, improves fidelity. Higher connectivity → better entanglement → higher fidelity → better hardware. Plus, better entanglement means more difficulty for classical algorithms to replicate. In terms of previously declared KPIs: ⚆ Advantage over classical solvers is present. ⚆ More efficient hardware implies opportunity for strides towards conservative energy use. Ultimately, focusing on crafting better hardware precedes and leads to finding practical applications. It is these incremental improvements that incrementally increase my already present faith in the technology itself. I look forward to seeing what practical applications the Quantinuum teams' efforts lead to -- congratulations to the team! #quantumcomputing #innovation

6

In this paper, Linyu Liu etal’s study the problem of uncertainty estimation and calibration for LLMs. We first formulate the uncertainty estimation problem for LLMs and then propose a supervised approach that takes advantage of the labeled datasets and estimates the uncertainty of the LLMs’ responses. Based on the formulation, they illustrate the difference between the uncertainty estimation for #LLMs and that for standard #ML models and explain why the hidden neurons of the LLMs may contain uncertainty information. They designed an approach which demonstrates the benefits of utilising hidden activations to enhance uncertainty estimation across various tasks and shows robust transferability in out-of-distribution settings. They distinguish the uncertainty estimation task from the uncertainty calibration task and show that a better uncertainty estimation mode leads to a better calibration performance. The method is easy to implement and adaptable to different levels of model accessibility including black box, grey box, and white box. The key is to develop an LLM which provides the user with a level of understanding of the likelihood that the results are good enough to make decisions, or not. https://lnkd.in/eYYHRZxz #Uncertainty #Estimation #LLMs #AI

9

1. 🚀 TextGrad uses text feedback for automatic differentiation, offering a new method for prompt optimization in large language models, effective in Q&A, molecular optimization, and radiotherapy planning. 2. 🔧 DSPy provides the algorithmic foundation for TextGrad to optimize LLMs' prompts and weights, reducing the need for manual adjustments. 3. 🌐 Automatic differentiation is crucial in deep learning, and TextGrad applies it to the text domain, optimizing through text feedback. 4. ✨ ProTeGi leverages natural language gradients and prompt editing to effectively optimize LLMs' prompts, finding the optimal configuration. 5. ❓ Despite TextGrad's innovation, its practical application may be limited by the performance of the teacher model and the need for extensive API calls. #AI #TextGrad #Innovation #DSPy #LLMOptimization #Algorithm #AutoDiff #DeepLearning #ProTeGi #PromptOptimization #AIOptimization #AIChallenges #TechLimitations

1