Dinesh Nair

Why has authentication become so complicated that there is an entire breed of Saas companies dedicated to it? Supertokens, Auth0, Clerk, Ory, Supabase, FusionAuth, Zitadel are just some of the companies out there offering 3rd party authentication services. "Auth is difficult and risky, don't do it yourself" is the generally accepted wisdom. But then I started looking into Clerk's under the hood implementation, and realized that they regenerate a new JWT token from their front end library every 60 seconds. No wonder they charge per "daily active users". So how did we end up there? 20 years ago you would just have a login, a password, generate a session id, store it somewhere (cache or database), put it in a cookie and be done with it. And a simple password recovery by email. Now, you have tokens, refresh tokens, OAuth 2.0, OICD and it's such a horrible mess. So much of a mess that there are companies that are basically selling a pretty looking form (with username, password, and Google Login button) on a recurring subscription. All of this because of "stateless auth". JWT tokens that can be verified independently of the service that issued them. But really, who needs stateless auth? Only the giants with countless (micro)services need that. Most of us do not. Most likely your service has a database with users anyway for other purposes, so your service is stateful in the first place and dependent on this database. The pain of having to manage and sync users in two different services is much bigger than the problem it solves. So just generate a damn session id, secured in a http only cookie. Store it hashed in your database, and cache it in memory in your application. It takes much less time to implement than researching auth vendors. And it is much easier to make sure it is secure than the tons of services and libraries you end up depending on for such a simple thing!

13

3 Comments

When you quantize down an LLM model, can you maintain its accuracy? I used to think no, but the data says otherwise. For example, Llama 3.1 405B FP16 has an average Accuracy score of 86.63. If you quantize down this model from FP16 to INT4, its Accuracy score falls to 86.47. That’s a recovery ratio of 99.81%. 🔥 🔥 In return, you get 4x more VRAM and higher performance. As people struggle to fit their models onto 8x H100s, a tiny loss in accuracy for 4x more VRAM sounds like a great bargain. Secret sauce seems to be the GPTQ algorithm, combined with neural networks being robust to quantization. Source: Neural Magic's Llama-3.1 70B and 405B INT4 models on Hugging Face. The link's in the comments. #opensource #llama3

35

5 Comments

There is a $7.5B company that: • Started with $0 funding • Has 100M+ active users • Generates $1B+ in ARR And now powering the open-source revolution. Here’s the story of GitHub: Tom and Chris were both working at CNET when they met in the mid-2000s. They were both passionate about software development and knew that developers needed better tools to manage and share code. In 2008, they started working on a side project that would change their lives. Git was a version control system at the time. But it had terrible UI and no functionality for collaboration. Their idea: build a web-based platform to host the Git repository. It would revolutionize the way developers collaborated on code. They launched GitHub in April 2008 with just a handful of users. Early on, they bootstrapped with an intense focus on customer experience and on exceptional UI. And they fully embraced the open-source community. The commitment paid off... Three years later, they had over a million developers on their platform. GitHub became the focal point of the open-source revolution. Hosting and collaborating on code was never so easy. Even big tech (FB, MSFT, Google etc) became customers. From there things skyrocketed 👇 By 2012, the biggest VCs started noticing. GitHub raised $100 million in Series A funding led by Andreessen Horowitz Their growth was explosive. By 2015, they had 9 million users and hosted over 21 million repositories. How did they scale so rapidly with a relatively small team? Simple answer: They leveraged talent all over the world. GitHub adopted a distributed team model early on, hiring talented developers from around the world. Scott Chacon, a co-founder, worked remotely from Mexico, setting the precedent for a global workforce. GitHub attracted top talent from regions like Eastern Europe etc. GitHub's distributed model also allowed for continuous development cycles with teams in different time zones. By 2018, GitHub had revolutionized the developer experience. And Microsoft bought them for $7.5 BILLION. All the co-founders are billionaires. Not bad for a side project. GitHub, WhatsApp, Wayfair and many giants were built on global talent. Trained pros take on rote execution at a fraction of the cost, while top teams focus on strategy and product. GrowthAssistant embeds elite, college-educated pros from the Philippines directly on your team. We find and place talent with the EXACT skillset you need. DoorDash, HubSpot, True Classic, and 200+ others use us Check it out:

23

2 Comments

Recently, there's been a lot of talk about Whisper, an open source speech-to-text engine developed by OpenAI. Here is how we use Whisper at Nabla: 1️⃣ The well-documented Whisper issues the article highlights are exactly why we spent 3 years and $5m, gathered and manually annotate a unique dataset of 7000 hours of medical encounters audio, and leveraged that dataset to train our own model. 2️⃣ Our model is based on Whisper, but it contains many improvements specifically developed to suppress hallucinations and make the transcription of medical terms more accurate than any off-the-shelf speech-to-text engine on the market. 3️⃣ In addition, the transcript is not directly included in the patient record. A second layer of processing extracts relevant data from the transcript. Each note produced is split into atomic facts, which are checked via an LLM query against the transcript and the patient context. Only facts for which we find definitive proof are considered valid. 4️⃣ After data is extracted, it is manually reviewed, edited and corrected if needed by the same clinician who did the encounter. Nabla also enforces that the review happens shortly after the encounter. 5️⃣ Each new version of our model undergoes rigorous evaluation by human professionals to ensure the documentation is both comprehensive and aligned with industry standards. 6️⃣ We have processed 9 million medical encounters in the past 24 months and got direct feedback from nearly 10,000 physicians about ways to improve Nabla. While some transcription errors were sometimes reported, hallucination has never been reported as a significant issue.

675

70 Comments

Check out how easy it is to gather all the data (backfill) you could ever need from Coinbase's BASE chain, using QuickNode ⚡Streams product 👉👇

9

1 Comment

It has been a crazy day and I wanted to share a click down on the various Llama repos we have so it’s clear where to go to get what. In short, there are five core repositories now including new repos for agentic development and for model level development. We’ve also consolidated the model level repos into a single repo making it much simpler to get things like basic inference code, the tokenizer, licenses, etc.. in a single place. llama-models: https://lnkd.in/gK4QWDXM - Central repo for the foundation models including basic utilities, model cards, license and use policies (currently private) PurpleLlama: https://lnkd.in/gsqkjqGM - Key component of Llama Stack focusing on safety risks and inference time mitigations llama-toolchain: https://lnkd.in/gj7NsU2d - Model development (inference/fine-tuning/safety shields/synthetic data generation) interfaces and canonical implementations (currently private) llama-agentic-system: https://lnkd.in/gb8M95Ck - E2E standalone Llama Stack system, along with opinionated underlying interface, that enables creation of agentic applications (currently private) llama-recipes: https://lnkd.in/gZiVTXxm - Community driven scripts and integrations As part of this effort, we are deprecating and consolidating the following model repos: - https://lnkd.in/gYrNHUVN (54.3k stars) - https://lnkd.in/d8gkkzCy (23.5k stars) llama-models is now the main Llama models repo. Enjoy!!

302

10 Comments

Currently, rollups are fragmented, leading to poor UX and disjointed liquidity. Here the first demo of Javelin by our portco NodeKit With Javelin, users can synchronously send messages and move funds across #rollups on NodeKit within a single 2s block.

11

LangGraph and Firecrawl seem so natural together FireCrawl has so many options available for smart LLM scraping, and structured output. With LangChain’s LangGraph as an orchestration layer you open a world of possibilities for structured output from unstructured web data - Its all you need 🕸️🔥 Combine FireCrawl & LangGraph 👇

17

2 Comments

Our dev vetting tech powers a $3B AI company. Yet we've never been more bullish on human developers. Here's why: In 2020, we bet big on data: spent hundreds of thousands analyzing 48M developer profiles to identify top talent. Why? Because we believed great developers would become more valuable, not less. That bet paid off: poolside AI (founded by GitHub's former CTO) licensed our technology to power their AI-driven developer assessment. Helping an AI developer company hire…human developers. This aligns perfectly with our vision: leveling the playing field so top developers outside main tech hubs can access the best opportunities - even in an AI-first world. Here's what we've learned: 1. AI is changing the game, but not how you think → It's not replacing developers, it's making great developers exponentially more valuable → The gap between good and great talent is widening faster 2. The real opportunity: → Building better ways to identify exceptional talent → Using AI to amplify human capabilities → Focusing on quality over cost arbitrage Back in July, we decided to start sharing our journey again. We're documenting our path from 2 developers to 300+, $25M in GMV, and the real insights behind building a transparent talent marketplace. Follow along here:

14

2 Comments

Today, OpenAI announced new automatic caching for Realtime API tokens, and an 80% price reduction for cached audio tokens. I updated our cost calculator spreadsheet, both to reflect the new cost reduction from caching and to revisit some assumptions about typical talk/turn times based on our logs. The realtime API keeps its own internal copy of the entire conversational history for a session. For each "turn" in the conversation the entire history, plus the newest user input, is used for inference. (You can see this in the usage metrics that the API reports.) - Total input audio for a typical 1-minute conversation (with three conversational turns and 30% non-speech time) will be just over a minute. - Total input audio for a typical 5-minute conversation will be ~26 minutes. - Total input audio for a typical 10-minute conversation will be ~100 minutes! Audio uses a lot of tokens — about 800 per minute of speech. So, without caching, costs add up a lot for long conversations. The new automatic token caching and 80% price drop for cached audio tokens is a big, big cost savings for long conversations. ~60% for a 5-minute conversation. ~70% for a 10-minute conversation. You can copy this cost calculator spreadsheet and edit it: https://lnkd.in/gAyjQRxi

74

10 Comments

Edge AI is the future! Are we prepared for the data privacy tornado that brings? Every device will have AI compute power and this will open up a new horizon for consumer (and business) applications. But who can reliably own the data? Businesses should build their own models now. #AIOwnership #DataPrivacy #EdgeAI

7

How long does it take to hire? This is where real data always helps! An example: We recently placed a Founding AI Engineer at an SF-based, seed stage company with a lot going for it: rising revenue, good signs of PMF, and repeat founder team. Kickoff was August 15th. That same day, we showed a potential candidate to them that could be a good fit. The candidate said yes to interviewing on August 16th. The first interview was on August 21st. And by August 29th they had completed multiple rounds and an offer was in progress. The candidate gave a verbal yes on September 2nd, and a final start date wasn't locked in until September 6th. Total time? 3 weeks end-to-end, and this is with a candidate introduction happening the second day after kickoff. Founders love to ask if we can compress hiring timelines further. My general response is: it can be challenging. Candidates want time to get to know a company. Companies need to time to complete a sufficient evaluation. And in cases where I've seen it pushed super-fast, candidates have balked and hiring managers have developed last minute concerns. So even when there's chemistry and a feeling of "fit-at-first-sight," good things can take time.

8

1 Comment

Stacked Diffs is the best kept secret of Meta and we copied it 💥 Stacking has been months into working at HyperTest. But now this is our default way to build and ship code because it does not block our devs for reviews. Devs love it and also complain why did we not implement this earlier. But why did we need to move to this? The traditional development workflow is this: 1, Finish a feature 2, Submit a PR 3, Await PR approval 4, Merge back into main once approved Once the PR is merged, start work on the next feature. This process takes a long time! The biggest time waster is that PRs sit waiting for review for hours or days. This wait becomes longer if current PR is dependent on a feature in another PR still under review This where stacking comes to rescue Stacking lets devs make many small PRs easily, without waiting for a review. To give a practical example, it’s pretty common that once a dev builds a feature, he wants to build the next feature using the completed one. With stacking, you don’t have to wait for the first feature to be approved and merged; you just keep working! The idea behind stacked diffs is that you can keep working on your main branch, and worry about reviews later. This is why very high performance teams, like the ones Meta, Google, Amazon, Rubrik and others use stacking as the only default way to write and ship code. But stacking is much easier to get started with in smaller teams. The learning curve was negligible but returns are great! We love it. #softwareengineering #stacking #git

16

A step-by-step guide to configuring the PawaPay SDK, from installation to handling API responses, ensuring perfect integration with your payment systems.

Haaaaampf... I have to migrate to using Gemini. The OpenAI API limitations are really problematic when you need to do complex long form writing. I need half a million input tokens for a complete operation on my end (at least). This involves dozens of Api calls, and multiple retrievals from the vector store. The Api limitations in Azure are forcing me to either start a doing load balancing to handle traffic, or migrate to using the Gemini API, which now has a *18 times bigger input window. The problem is that my entire RAG system was custom built on Azure to use the OA deployment feature. So now I have to rewrite and migrate to GCP. Good time to dockerize and stop being vendor locked. Any suggestions? maybe I'm missing something.

10

11 Comments

🔔 New Full Stack Development Project: Roblox, San Mateo, California Seeks Full Stack Development Partner for Payments Optimization Estimated Budget: $150K-$200K --- Project Requirements a. Payments Optimization b. Full Stack Development c. Data Processing and Analysis Technical Skills Required - Scalable, distributed systems - Large-scale, data-driven architecture - API design - SQL / NoSQL data storage - Modern front-end development (React, TypeScript) - Data processing technologies (Spark, Flink, Hive, Airflow, Kafka) - Strong computer science fundamentals and software engineering principles --- See the PDF for details about this opportunity. Join our group and connect with 5K web agency owners in our community. 👉 https://lnkd.in/g5hP2FU --- 🟢👀🟢 If the above project is not the perfect fit for your agency ... I recommend getting clients from inbound channels. (I cover multiple proven lead gen strategies in my program) And they are often counterintuitive. For example: Agencies are generating $30k+/mo getting clients from Upwork (Web Dev Agency Case Study) Some agencies in my mastermind are getting $10k - $30k web dev leads from Upwork, check out one of our latest case studies: 👉 https://lnkd.in/eemDj4yZ ($45k/mo Case Study) The problem: Usually the projects on Upwork are too small, the competition is too high and it takes too much time. The solution: So we combined all our knowledge from the agencies in my mastermind group who have the most experience succeeding on Upwork. After many tests we finally came up with a powerful new method of bidding & winning larger deals on Upwork. We broke the process into 3 stages, found the bottlenecks and used our combined experience to optimize each stage: (1) the selection stage (2) the bidding stage (3) the booking and closing stage. This gave us over a 900% increase in results from Upwork -- and most of the steps can be automated. As you can see it's working very well: https://lnkd.in/eemDj4yZ If you want to get the full strategy, DM me with "Upwork" 👉 https://m.me/upscalemethod and I'll be happy to discuss this in more detail. --- Chris "Upscale" Johnson Web Dev & WordPress Leads Web Agency Coach UpscaleMethod . com #webdesignleads #webdesignagency #webdesign #webdesigner #webdeveloper #webagency #digitalagency #agencycoach #webagencycoach #digitalagencycoach #mentor #businesscoach #mastermind #mentorship

1

In-App Subscriptions - Key Trends in 2024 🚀 Adapty.io latest report on in-app subscriptions is out, revealing crucial trends. Highlights: 1. One of the highly discussable trends now is switching to weekly subscriptions. Our data shows that the weekly subscription market share is over 55% in 2024, up from 36.8% in 2022. 2. We tracked price changes for weekly, monthly, and annual plans, comparing the US, EU, LATAM, and APAC regions. Since 2022, the largest increase in weekly subscription prices was seen in the EU (+7.5%), and the largest decrease was in LATAM (-3.3%). 3. App developers see up to 100x more revenue with paywall experiments. In the US, app owners run 3x more pricing tests than elsewhere. The report shows which paywalls have the highest LTV. Read the full report for free → https://lnkd.in/dFAmz93q

14

1 Comment