Alex Miller

A la IA se le acaban los datos…avanzaremos hacia la mejora del Razonamiento lógico … …” On a positive note, he suggested this shift toward real reasoning could help reduce hallucinations, as future AI systems might use logical thinking and self-reflection to verify and correct their own statements—capabilities that current systems, which rely mainly on pattern recognition and intuition, don't have…. “

5

2 Comments

When you quantize down an LLM model, can you maintain its accuracy? I used to think no, but the data says otherwise. For example, Llama 3.1 405B FP16 has an average Accuracy score of 86.63. If you quantize down this model from FP16 to INT4, its Accuracy score falls to 86.47. That’s a recovery ratio of 99.81%. 🔥 🔥 In return, you get 4x more VRAM and higher performance. As people struggle to fit their models onto 8x H100s, a tiny loss in accuracy for 4x more VRAM sounds like a great bargain. Secret sauce seems to be the GPTQ algorithm, combined with neural networks being robust to quantization. Source: Neural Magic's Llama-3.1 70B and 405B INT4 models on Hugging Face. The link's in the comments. #opensource #llama3

35

5 Comments

NVIDIA's Jensen said "ChatGPT democratized computing, Llama2 democratized generative AI" ...and now Llama3 is the next level 🔥 It's fascinating to watch Meta's strategic moves. With Llama3, they prioritized training on a massive 15T token dataset to pack all into a lean 70B param model, instead of building a massive model. This allows Llama 3 to match trillion+ parameter models like GPT-4, but at 1/10th the compute, storage and inference costs! 💰 This technique was published by Google DeepMind a few years ago https://lnkd.in/gx7VU8aA Meta is not a dark horse in AI anymore. They might be the top dog. https://lnkd.in/gRRJYuxt #llama2 #llama3 #llm #chatgpt #gpt4 #ai #ml

16

5 Comments

There's now no shortage of options for open source model hosting providers. i.e., take your pick for Llama 3 (Groq, Together AI, Telnyx, etc.). Am I wrong in thinking the only real difference now between them is speed, cost and readability? If so, this space just became commoditized quickly and reminds me of the telco provider wars where its a race to the bottom on price. Which means the cost of running AI applications is approaching something close to zero... 🤯 #ai #cx

14

2 Comments

After lots of feedback and self-testing, I decided to re-write my neovim distribution, clax.nvim. 🔗 https://lnkd.in/gq4CjFMY The rewrite has been inspired by josean martinez's (https://www.josean.com/) config. So what things does the rewrite offer this time? ✅ user-friendly bash script for instant setup ✅ small tweaks including relative line numbers ✅ better syntax highlighting ✅ catpuccin mocha/latte colorscheme (with transparency) ✅ markdown preview feature Here's some more things that will be added soon: ⚪ bracket autocompletion ⚪ status-line theme ⚪ transparency for splits Thanks to Yash Mehrotra Kashiful Haque Aditya Siddharth Jyoti Deepesh Kumar Dawar and others from Google Student Developer Clubs - IIT Madras's Linux community for their feedback.

34

4 Comments

~40% of Frec’s customers are engineers working in tech, many of whom expect frequent and large capital gains. They also prefer to self-manage their money, which makes Frec an ideal platform. Meet Evan: an engineering leader who upgraded from investing in ETFs to direct indexing. Direct indexing appealed to Evan because it offers comparable returns to his ETFs while saving him money on advisory and ETF fees. “ETFs are true and tested, and giving folks a different way of investing in the market while saving more money is really appealing.” He’s used many other robo-advisors and other fintech tools, but “Frec seems to maximize tax loss harvesting opportunities better than any other product that I’ve used.” He also likes Frec's transparent fee structure.  “I feel like with a lot of other companies, that’s muddled in with the cost of investments,” he told us, “and you don’t really know how much you’re paying or how much you’re not saving as a result. I think Frec does a really good job of laying all of that out and giving you a true understanding of the cost of these investment products.” Read more about his Frec journey here ⬇

45

2 Comments

There is a $7.5B company that: • Started with $0 funding • Has 100M+ active users • Generates $1B+ in ARR And now powering the open-source revolution. Here’s the story of GitHub: Tom and Chris were both working at CNET when they met in the mid-2000s. They were both passionate about software development and knew that developers needed better tools to manage and share code. In 2008, they started working on a side project that would change their lives. Git was a version control system at the time. But it had terrible UI and no functionality for collaboration. Their idea: build a web-based platform to host the Git repository. It would revolutionize the way developers collaborated on code. They launched GitHub in April 2008 with just a handful of users. Early on, they bootstrapped with an intense focus on customer experience and on exceptional UI. And they fully embraced the open-source community. The commitment paid off... Three years later, they had over a million developers on their platform. GitHub became the focal point of the open-source revolution. Hosting and collaborating on code was never so easy. Even big tech (FB, MSFT, Google etc) became customers. From there things skyrocketed 👇 By 2012, the biggest VCs started noticing. GitHub raised $100 million in Series A funding led by Andreessen Horowitz Their growth was explosive. By 2015, they had 9 million users and hosted over 21 million repositories. How did they scale so rapidly with a relatively small team? Simple answer: They leveraged talent all over the world. GitHub adopted a distributed team model early on, hiring talented developers from around the world. Scott Chacon, a co-founder, worked remotely from Mexico, setting the precedent for a global workforce. GitHub attracted top talent from regions like Eastern Europe etc. GitHub's distributed model also allowed for continuous development cycles with teams in different time zones. By 2018, GitHub had revolutionized the developer experience. And Microsoft bought them for $7.5 BILLION. All the co-founders are billionaires. Not bad for a side project. GitHub, WhatsApp, Wayfair and many giants were built on global talent. Trained pros take on rote execution at a fraction of the cost, while top teams focus on strategy and product. GrowthAssistant embeds elite, college-educated pros from the Philippines directly on your team. We find and place talent with the EXACT skillset you need. DoorDash, HubSpot, True Classic, and 200+ others use us Check it out:

23

2 Comments

Recently, there's been a lot of talk about Whisper, an open source speech-to-text engine developed by OpenAI. Here is how we use Whisper at Nabla: 1️⃣ The well-documented Whisper issues the article highlights are exactly why we spent 3 years and $5m, gathered and manually annotate a unique dataset of 7000 hours of medical encounters audio, and leveraged that dataset to train our own model. 2️⃣ Our model is based on Whisper, but it contains many improvements specifically developed to suppress hallucinations and make the transcription of medical terms more accurate than any off-the-shelf speech-to-text engine on the market. 3️⃣ In addition, the transcript is not directly included in the patient record. A second layer of processing extracts relevant data from the transcript. Each note produced is split into atomic facts, which are checked via an LLM query against the transcript and the patient context. Only facts for which we find definitive proof are considered valid. 4️⃣ After data is extracted, it is manually reviewed, edited and corrected if needed by the same clinician who did the encounter. Nabla also enforces that the review happens shortly after the encounter. 5️⃣ Each new version of our model undergoes rigorous evaluation by human professionals to ensure the documentation is both comprehensive and aligned with industry standards. 6️⃣ We have processed 9 million medical encounters in the past 24 months and got direct feedback from nearly 10,000 physicians about ways to improve Nabla. While some transcription errors were sometimes reported, hallucination has never been reported as a significant issue.

675

70 Comments

Big day - I just launched Portfolio Analyst at vectabass.com The past year of building VectaBass has been a wild ride. What started as a platform for hosting LLM agents has evolved into something I'm really excited about. After countless late nights (and maybe too much Monster), I've built something I think could change how people interact with web apps and in this case, investing. Portfolio Analyst is simple: talk to an AI about your portfolio, get instant analysis, and understand what the numbers actually mean. Want to optimize your portfolio? Check the efficient frontier? Understand your risk exposure? Just ask. All the heavy lifting - from pulling market data to running complex calculations - happens behind the scenes. I've been quietly working on this for months, backed by some amazing people who saw the potential early on. Their support means I can finally focus on this full-time and really accelerate development. It's not perfect yet. This first version is rough around the edges. But it's real, it works, and it's solving a problem I've personally struggled with - making institutional-grade portfolio analysis accessible and actually understandable. If you're interested in trying it out, head over to vectabass.com. I'd love to hear what you think. This is just the start, and your feedback will shape where it goes next. To everyone who's supported this journey so far - thank you. You know who you are. I'm looking forward to working on more projects as time goes on. Now that the hardest part is behind, the platform I've built so far is going to be an exciting spring board to launch into so many ideas. Automated search engines, document generation, swarms and more! #FinTech #LLM #Agentic

29

2 Comments

Just sent out the latency newsletter for April. Noticing some really interesting trends. 🏆 Claude Instant regains the top spot after losing it for only a single month 🐢 Claude Sonnet + Haiku both saw a major increase in latency (~35% on average). This pattern of a slow down in performance the month after launch happened to GPT-4-Turbo as well. Could be related to increased adoption as people had more time to test and implement. 🐇 A model that I really like but doesn't get much attention, gpt-3.5-instruct, saw the biggest gains ( >50%) Plus a whole bunch more Get access to all the data via the link below. One email, once a month, packed with actionable and data-backed recommendations.

9

3 Comments

A lot of important answers could be like low hanging fruit, and properly trained LLMs can probably reach them. My models on AdamEdsall.com and different ones through Allez and Connext Marketing have specialized areas to ensure success, though. More mechanical, rigid things going on than seeing what you get back from predefined training sets with a query. #GenieInALampParadox #Singularity #AI

1

3 Comments

People often say, "Platform engineering is a socio-technical discipline that demands a product and people-oriented approach." But what does this mean for you and your organization? I have an exceptional team Jennifer Riggins Bryan Ross from GitLab, Colin Griffin from Krumware, Ramiro Berrelleza from Okteto Jorge Turrado Ferrero ready to share insights on how to champion platform engineering initiatives, where to begin, and how to structure your team effectively. Platforms promise to enable application developers, data scientists, security teams, and other stakeholders to work more efficiently and effectively, but what about the Intricacies and Pain Points of Platform Engineering Initiatives (Balancing Innovation and Stability), (Scalability and Performance), (Computational balance), (Developer Experience and Onboarding) Striking the right balance between introducing new features and maintaining a stable, reliable platform is a common challenge - How can we ensure our platform is both innovative and stable? As organizations grow, their platforms must scale to handle increased loads without sacrificing performance. - What strategies can we implement to ensure our platform scales efficiently and maintains high performance? Efficient resource management is vital for reducing costs and improving performance. - How can we optimize the use of our computational, storage, and networking resources? A seamless onboarding experience and robust user support is essential for user satisfaction and platform adoption - What can we do to ensure a smooth onboarding experience for new users and provide ongoing support? Save the link and hit subscribe: https://lnkd.in/dMfgkdFX #platformengineering #platformorchestration #platformascode #developerexperience #devex #infraautomation #idp #developerportals

13

1 Comment

I have yet to see any of the "LinkedIn experts" concisely/accurately describe what the blockchain is and is for, so here: it's an append-only DB of crypto primitives for zero-trust synchronization. That's it. Want some simple analogies to compare it to? Decentralized supabase with omni read access and RLE writes. The writes are announced just like Kafka streams with a deliver-once subscriber guarantee. Want it even simpler? A GitHub repo where writes are bot-validated, auto-accepted PRs that guarantee the contents and the originating author’s identity. Want it non-technical? A shared, structured google doc. Everyone can read it, and anyone can sign up to write new rows to it. The new rows have to be at the bottom, and existing rows can't be deleted or edited. When someone signs up, they get a set of login credentials which confirm their identity whenever they add rows to it. You can also set special validation rules and restrictions on what information can be added and the conditions a new row must meet to be accepted. You can add notes to it via something called "a soft fork" (members don't have to agree to it). You can edit the validation rules and restrictions via something called a "hard fork" (most members with login credentials must agree to the rule/restriction changes). No online courses, seminars, or consultants needed. And yes, it's useful for AI.

6

4 Comments

Devin - This is where the code agents hype started! Now it is generally available. Teams plan only $500 per month. Costing? They introduce ACU (Agent Compute Unit) - roughly 15 mins of Devin work.  Glad we are moving away from token counts to coarser measures of compute. Let's see how well do the open source code agents like All Hands AI compare with Devin and how soon do they catch up. --- Like this post? Discover more at offnote.substack.com/notes

17

1 Comment

Quite informative article on tokens, tokenizers and their impact on #llm performance and cost. Recommend a read.

4

I do a lot of AI automation work, much of it involving batch processing and reprocessing of vast amounts of text. The low cost and high quality of Claude 3 Haiku (25 cents per million tokens in and $1.25 per million tokens out) has been fantastic. But today, OpenAI released GPT-4o-mini for use in API calls, driving the cost of AI automations even lower: 15 cents per million tokens in and 60 cents per million tokens out! To give you a sense of the savings: I once accidentally ran my batch processor using the high-end GPT-4o model. With 12 million tokens submitted and 2.5 million tokens returned, it cost me $97.50. With GPT-4o-mini, today that cost is only $3.30. That’s over a 96% savings! 😲 I have reviewed the Llama 3 8b + 70b Meta open-source models for these automations. The cost is lower still (the model is free, but hardware time is not). However, in my tests, the quality just isn't the same. Plus, considering the additional effort to host an API to access the Llama 3 models, my engineering time and money have so far been better spent using the low-cost models from OpenAI and Anthropic. Anyone else finding similar results in savings + quality?

35

1 Comment