Donny Greenberg

Wonderful article about Emily Knight and the whole The Engine Accelerator team on the amazing work their doing changing the world with #ToughTech!

28

LLMs are trained on internet-scale data. But to make AI work for work, models need to be grounded in your enterprise data -- a much harder problem than meets the eye! How do you connect to all your data sources and ingest them frequently enough while managing compute/storage costs? How do you make sure that you only show users results from documents they have access to? How do you rank results without internet-scale click and link data? How do you process the content of documents that are often hidden in PDFs and presentations? These are tricky problems, and it's why enterprise search didn't work for nearly a decade, until Glean came along. Turns out it's the exact same set of challenges needed to give LLMs the right context to work their magic on your enterprise content. Glean spent 5 years building the ingestion, search, and ranking infra needed to solve this problem well. This has given them the right to run the "Work AI" race and delight users at work. CEO Arvind Jain shares more on the latest episode of Training Data. Listen here, or wherever you get your podcasts: YouTube: https://seq.vc/ytd Apple: https://seq.vc/pit Spotify: https://seq.vc/jnz 

203

10 Comments

🎯Data was, is, and will be the only fuel. Analogy with the fossil fuel is a good one, since the data we have today is as dirty as oil and coal. What big tech companies tried is to put a lot of "garbage" data, and what they did get was fantastic. Yet, we are at the point of diminishing returns. Data is the beginning and the end of this cycle.” Great thread, particularly this comment 👆from Ilija Lazarevic

20

2 Comments

Kim Burgaard Rachel Perez Jay Leverett David Bellotti -- I'll be listening to this one for sure! Sergio H. Serge Madenian Episode Summary: The episode features an in-depth interview with Mike Krieger, the new chief product officer at AI company Anthropic, discussing his background, the decision to join Anthropic, the company's focus on responsible AI development, the challenges and opportunities in building AI products, and his vision for the future of AI technology and its applications. https://lnkd.in/guWW8cku

3

Today, I found a fascinating way to think about “Tokens” in LLMs. It came from a conversation between Jensen Huang and Patrick Collison at a recent Stripe event. When asked about the future of compute capacity in five years, Jensen smartly dodged that question a bit, but then gave an excellent explanation and analogy about why tokens are the new forces that will power the next decades of humanity. He explained that we're now producing something unprecedented (and at scale): floating point numbers that possess “value”, which we now call tokens. These tokens are valuable because they encapsulate “intelligence”. People are now taking these tokens and transforming them into English, French, images, videos, chemicals, proteins, robotic movements, etc. And many are working hard to expand the range of concepts and ideas we can create with these tokens. He then goes on to made a compelling comparison between tokens and electricity. In the past industrial revolution, we successfully found a way to convert “atoms” into “electrons” (by boiling water to power electricity turbines). And now, we've discovered a way to convert "electrons" into "tokens" (by using energy to power data centers that train and run LLMs). When electricity was first introduced, few people understood its value. Today, paying for kilowatts is routine. The same will happen with tokens. Right now, only early adopters and builders are paying tokens. Soon, everyone will be paying for tokens on a daily basis to supercharge productivity and power new products and services. Many new industries will be born from and built on top of tokens. When I first heard about this way of thinking, I got goosebumps. Perhaps it’s because Jensen has a good storytelling skill that helps him sell what he builds, but I have to admit this way of thinking gave me a brief surge of pride. I'm proud of humanity's collective effort and creativity, which have taken us from living under rocks to building machines that can "think." That's nothing short but a miracle.

18

5 Comments

Starting this Monday, Oct 7, on the first Monday of each month, CryptoMondays NYC will focus on the intersection of crypto and Wall Street (i.e. institutional participation in crypto assets). Please join us Monday as we kick off this initiative with a Fireside Chat with Zach Pandl, Head of Research at Grayscale Investments. Grayscale's iconic Bitcoin Trust, started in 2013, peaked at over $40B in AUM in 2021. Today, Grayscale offers more than 20 crypto trusts (e.g. ETH, Bittensor, & Solana) totaling more than $20B in AUM. Prior to joining Grayscale, Zach was the Senior Economist/Macro Strategist at Goldman Sachs. I'm looking forward to seeing lots of my Wall Street friends who range from curious about crypto, to obsessed. Anthony Scaramucci, Alexander Golding, Anoop Kansupada, Anthony Pompliano Catherine Wood Yassine Elmandjra Dave Balter Hadley S. Thomas (Tom) Lee Matthew Walsh Miko Matsumura Pete Najarian Mo Shaikh Anna Bertha Evan F. Seth Ginns Jalak Jobanputra James Parillo Mark Goldwasser Keli Callaghan Mike Chan Nolan Ford Max Shapiro, CFA, CPA Sunny Parikh Ashim Mehra Wendi Dritz Carver Tyler Carter Michael Gold Mike McGlone Matt Wright Marcy Simon Casey Taylor Brad Feinstein Jonathan Steinberg Gil Rosen Rick Bradt Kenneth Kruger Carl Vogel #bitcoin @eth #btc #crypto #cryptocurrency #web3 #AI https://bit.ly/CM1007ZP

64

5 Comments

How 𝘾𝙖𝙪𝙨𝙖𝙡 𝘼𝙄 Can Help Reduce Hallucinations: New Q&A with Jayeeta Putatunda I recently spoke with Jayeeta, Lead Data Scientist and Director at Fitch Group Inc, about using causal AI to ground language models in real relationships, not just correlations: 𝘛𝘩𝘪𝘯𝘬 𝘰𝘧 𝘧𝘪𝘯𝘢𝘯𝘤𝘦 𝘰𝘳 𝘢𝘯𝘺 𝘥𝘰𝘮𝘢𝘪𝘯 𝘸𝘩𝘦𝘳𝘦 𝘺𝘰𝘶 𝘤𝘢𝘳𝘦 𝘢𝘣𝘰𝘶𝘵 𝘤𝘢𝘶𝘴𝘦 𝘢𝘯𝘥 𝘦𝘧𝘧𝘦𝘤𝘵 𝘳𝘢𝘵𝘩𝘦𝘳 𝘵𝘩𝘢𝘯 𝘫𝘶𝘴𝘵 𝘤𝘰𝘳𝘳𝘦𝘭𝘢𝘵𝘪𝘰𝘯. 𝘌𝘤𝘰𝘯𝘰𝘮𝘦𝘵𝘳𝘪𝘤𝘴 𝘩𝘢𝘴 𝘢𝘭𝘸𝘢𝘺𝘴 𝘥𝘰𝘯𝘦 𝘵𝘩𝘢𝘵, 𝘵𝘳𝘺𝘪𝘯𝘨 𝘵𝘰 𝘴𝘦𝘦 𝘪𝘧 𝘢 5% 𝘤𝘩𝘢𝘯𝘨𝘦 𝘪𝘯 𝘱𝘢𝘳𝘢𝘮𝘦𝘵𝘦𝘳 𝘈 𝘭𝘦𝘢𝘥𝘴 𝘵𝘰 𝘴𝘰𝘮𝘦 𝘴𝘩𝘪𝘧𝘵 𝘪𝘯 𝘱𝘢𝘳𝘢𝘮𝘦𝘵𝘦𝘳𝘴 𝘉 𝘰𝘳 𝘊. 𝘠𝘰𝘶 𝘤𝘢𝘯 𝘳𝘶𝘯 𝘴𝘪𝘮𝘶𝘭𝘢𝘵𝘪𝘰𝘯𝘴 𝘵𝘰 𝘴𝘦𝘦 𝘪𝘧 𝘵𝘩𝘢𝘵 𝘭𝘪𝘯𝘬 𝘩𝘰𝘭𝘥𝘴, 𝘰𝘳 𝘪𝘧 𝘵𝘩𝘦𝘳𝘦’𝘴 𝘫𝘶𝘴𝘵 𝘢 𝘭𝘰𝘰𝘴𝘦 𝘤𝘰𝘳𝘳𝘦𝘭𝘢𝘵𝘪𝘰𝘯. 𝘕𝘰𝘸 𝘤𝘰𝘮𝘣𝘪𝘯𝘦 𝘵𝘩𝘢𝘵 𝘸𝘪𝘵𝘩 𝘨𝘦𝘯𝘦𝘳𝘢𝘵𝘪𝘷𝘦 𝘈𝘐. 𝘐𝘧 𝘺𝘰𝘶 𝘧𝘦𝘦𝘥 𝘵𝘩𝘦 𝘨𝘦𝘯𝘦𝘳𝘢𝘵𝘪𝘷𝘦 𝘮𝘰𝘥𝘦𝘭 𝘷𝘢𝘭𝘪𝘥𝘢𝘵𝘦𝘥 𝘤𝘢𝘶𝘴𝘢𝘭 𝘮𝘢𝘱𝘴 𝘰𝘳 𝘳𝘦𝘭𝘢𝘵𝘪𝘰𝘯𝘴𝘩𝘪𝘱𝘴, 𝘵𝘩𝘦𝘯 𝘸𝘩𝘦𝘯 𝘪𝘵 𝘱𝘳𝘰𝘥𝘶𝘤𝘦𝘴 𝘢 𝘳𝘦𝘱𝘰𝘳𝘵, 𝘪𝘵’𝘴 𝘭𝘦𝘴𝘴 𝘭𝘪𝘬𝘦𝘭𝘺 𝘵𝘰 𝘪𝘯𝘷𝘦𝘯𝘵 𝘧𝘢𝘭𝘴𝘦 𝘤𝘰𝘯𝘯𝘦𝘤𝘵𝘪𝘰𝘯𝘴. 𝘐𝘧 𝘺𝘰𝘶’𝘳𝘦 𝘴𝘶𝘮𝘮𝘢𝘳𝘪𝘻𝘪𝘯𝘨 𝘩𝘰𝘸 𝘈𝘱𝘱𝘭𝘦’𝘴 𝘨𝘳𝘦𝘦𝘯 𝘪𝘯𝘪𝘵𝘪𝘢𝘵𝘪𝘷𝘦𝘴 𝘭𝘦𝘥 𝘵𝘰 𝘢 𝘳𝘦𝘥𝘶𝘤𝘵𝘪𝘰𝘯 𝘪𝘯 𝘤𝘢𝘳𝘣𝘰𝘯 𝘦𝘮𝘪𝘴𝘴𝘪𝘰𝘯𝘴, 𝘺𝘰𝘶 𝘤𝘢𝘯 𝘷𝘢𝘭𝘪𝘥𝘢𝘵𝘦 𝘵𝘩𝘢𝘵 5% 𝘰𝘳 10% 𝘧𝘪𝘨𝘶𝘳𝘦 𝘵𝘩𝘳𝘰𝘶𝘨𝘩 𝘺𝘰𝘶𝘳 𝘤𝘢𝘶𝘴𝘢𝘭 𝘮𝘰𝘥𝘦𝘭. 𝘛𝘩𝘦 𝘨𝘦𝘯𝘦𝘳𝘢𝘵𝘪𝘷𝘦 𝘮𝘰𝘥𝘦𝘭 𝘤𝘢𝘯 𝘵𝘩𝘦𝘯 𝘨𝘦𝘯𝘦𝘳𝘢𝘵𝘦 𝘵𝘦𝘹𝘵 𝘵𝘩𝘢𝘵 𝘳𝘦𝘧𝘭𝘦𝘤𝘵𝘴 𝘵𝘩𝘰𝘴𝘦 𝘳𝘦𝘭𝘢𝘵𝘪𝘰𝘯𝘴𝘩𝘪𝘱𝘴, 𝘳𝘢𝘵𝘩𝘦𝘳 𝘵𝘩𝘢𝘯 𝘨𝘶𝘦𝘴𝘴𝘪𝘯𝘨. Jayeeta also shares her path from econometrics to NLP innovations, her perspective on the rapid evolution of AI technologies, and the importance of bridging research with real-world industry challenges. Other takeaways from our discussion: • Finance needs specialized frameworks beyond generic AI metrics • Regulators require clear evidence chains, not black boxes • Traditional statistical models + new AI techniques are key to adoption The full interview is out on Thursday. Subscribe to AI Street to get in your inbox. Link in the first comment. Follow me (Matt Robinson) and AI Street for more on AI + Finance.  

27

3 Comments

Something that stuck with me during my interview with Naveen Rao, VP of Generative AI at Databricks: "I take the hard path." He built an AI-focused chip company Nervana in 2014 when machine learning was still academic theory and the chip industry was deeply out of favor. Naveen saw computing architecture would need to fundamentally change for ML workloads. And then in 2019, observing the power of LLMs like BERT, Naveen foresaw the need for enterprise-grade services to manage their training, deployment, and customization. That vision led to a $1.3B acquisition by Databricks in 2023. What I love about Naveen's approach is his willingness to build for markets that don't exist yet. "There isn't a market, employees don't even know about it. So how the hell do you build a company?" You educate. You convince. You get people to see what you see. It's brutal work, but it's important work. You're reshaping how people think about problems.

176

8 Comments

ex Google CEO Eric Schmidt spoke at Stanford this week. Lots of nuggets here, but most interesting for the near term, Schmidt believes in the next year AI is set to make a big leap forward. He predicts AI will combine three important features: 1. Vast knowledge retention (1 million token models) 2. Text-to-action capability 3. Personal AI agent fleets While the full impact is unknown, everyone may soon command their own AI teams. Here's an example: Imagine you're developing new quantum computing algorithms. Your AI assistants could: - Continuously scan and synthesize global quantum research papers - Generate novel algorithm ideas based on this knowledge - Simulate and test these algorithms across various quantum architectures - Automatically document findings and prepare draft papers - Identify potential collaborators and draft outreach emails All initiated by a command like "Advance our quantum algorithm research." This could dramatically accelerate R&D cycles and scientific breakthroughs.. https://lnkd.in/eJbg6cnD #frontiertech #AI #futurefrontier

22

8 Comments

In retrospect, 2024 was a pretty amazing year for innovation out of University of California, Berkeley. My Top 10 highlights below. Any I missed? 1. Databricks secured the largest VC round ever—a $ 10B funding round, led by multiple Berkeley founders. 2. OpenAI raised the 2nd largest VC funding round ever - $6.5B. Co-founded by Berkeley alum John Schulman. 3. Since launching Built By Berkeley (a newsletter that tracks Berkeley founded companies funding) in July 2024: 229 companies raised 27.4B. ~$1.1B per week raised by Cal founders! Wild! 4. Out of the 13 2024 Nobel Prize laureates - 4 had Berkeley connections. 5. Record-breaking Berkeley Olympic performance: 5 Gold, 10 Silver, 8 Bronze medals. More than many countries!  6. Berkeley-founded Perplexity emerged as a genuine Google Search competitor, reaching a $9B valuation (9x increase in a year) with 15M+ active users (probably already much higher than this). Along with some unbelievable execution speed from Aravind Srinivas and team. 7. Chatbot Arena, founded at Berkeley, became the industry standard for AI model benchmarking. 8. Rich Lyons became the new Berkeley chancellor. Rich will be a huge innovation catalyst for Berkeley. 9. Berkeley beats Stanford University 21-24 in the Big Game…not innovation but fun nevertheless.  10. Berkeley leads the PitchBook rankings with 1,642 VC-backed undergrad alumni-founded companies. 245 more than second-placed Stanford. Berkeley is shaping up to be ground zero of the coming AI wave from a company formation and research perspective. Looking forward to a bigger and better 2025.

1,812

71 Comments

Batch processing for LLM inference activity: "One sign of that is the rise of batch processing. Today, popular consumer AI products like ChatGPT and Perplexity provide users with near instantaneous responses, otherwise known as real-time inference. However, businesses don't always need immediate responses and are often willing to wait hours, days, or even weeks for responses, as long as they don't have to shell out as much money. Founders of cloud and inference providers have told me there's growing pressure from business customers for this kind of flexibility, otherwise known as batch processing." -- Nathaniel Whittemore on "The AI Daily Brief" (with much deserved shout-out to his inspo from Stephanie Palazzolo at The Information) I fall into this bucket as we have a lot of backfilling to do as we're building out the underlying data in our service, and some of that could just as well be done in the wee hours of the night. The new stuff flowing in I want processed ASAFP, but these backfills—I'd mos def take more cost-efficient delayed batch processing. There's also use cases like needing (near) real-time inference for the first inference pass for timeliness / contemporaneousness value, but then a second pass slow-time inference for wider classification exercises over an ever-growing data corpus. I could see us mixing Sonnet & Haiku models in both cases. Just a heads up to Anthropic Mike Krieger as you contemplate this, along with Amazon Web Services (AWS) Jeff Barr Mark Roy Swami Sivasubramanian on the Bedrock side of things...and I don't mean the "batch" mode that doesn't seem to have any real value other than helping AWS manage its own rate limit challenges. (DeepCast is using Bedrock.)

11

2 Comments

For all you AI and data needs this is worth being aware of.

4

Predictions for the future of LLMOps are varied.   But what is clear from speaking to Clemens Rawert, Co-Founder at Langfuse;   Is that users have become a lot more thoughtful and systematic as LLMs have matured.   And building golden datasets are going to be even more important for quality benchmarks.   Just 2 points of importance at Langfuse (YC W23), Take a look at the video to find out what else.... 💡

6

🎃 Ready to carve out your path to Product-Market Fit?🎃 Join Companyon Ventures’ Behind the Curtain roundtable tomorrow (Oct 30) with Michael Ho, Firas Raouf, Eric Morgan, and Hersh Tapadia as they 'exorcise' the mysteries of PMF. Don’t ghost us—RSVP in the comments 👻 https://lnkd.in/emCJcn4w #emergingmanager #venturecapital #productmarketfit

46

2 Comments

I like this idea of "technical taste". It gets into where software engineering is as much creative art as it is science. Four great takeaways (IMHO) for software devs in this era in particular: 1. Aspiring engineers should cultivate a sense of curiosity, experiment with different tools and technologies, and embrace a mindset of continuous learning. 2. AI has the potential to streamline processes and enhance productivity for engineers, but it may also lead to disruptions in traditional software development workflows. 3. Developing technical taste and judgment is essential for making informed decisions about which technologies and approaches to pursue. 4. Collaboration and open-mindedness are key to leveraging the full potential of AI and staying ahead of technological advancements. I also really liked these two quotes from Sam Schillace: 1. "The right time to do something is when you have that feeling in the pit of your stomach that's like, 'oh, this is a great idea and it's going to suck to build because nothing's ready yet.'" 2. "Technical taste is like, 'how well have you consolidated that set of experiences and heuristics into judgment that you can apply accurately when you see new things?'" 3. "It may be the case that very small teams can do very large projects, or like we were talking about before, it may be the case we're just going to get really ambitious about what we try to do with the same size of teams, which is kind of where I would put my money." Great job on continuing to put out great episodes, Brett Berson and team First Round Capital! #ai #engineering #softwaredevelopment

5

🚨🚨 🚨 Unlocking Innovation: The Power of the Adjacent Possible Innovation programs can stall when the team tries to ideate way beyond their collective perspective and frame of reference. Like a software engineer brainstorming genetic modification of mycelia, or a senior executive ideating process improvements which they haven't been participated in for decades. It's challenging, exhausting, and ultimately self-defeating. Enter the concept of the "Adjacent Possible" - a game-changer for innovation. What is it? The Adjacent Possible, coined by Stuart Kauffman, represents the realm of new ideas within reach based on our current knowledge, resources, and capabilities. Why it matters: ➡️ Realistic Innovation: it focuses on achievable advancements, not far-fetched concepts. ➡️ Incremental Progress: each innovation opens doors to new possibilities. How to leverage it: • Constantly explore the boundaries of your adjacent possible. What's in your line of sight is what you know best. • Combine existing ideas in novel ways. Even small perturbations from status quo can generate large innovation leaps. • Stay curious about developments in adjacent industries. Cross-domain patterns create fresh sources of ideas, while also expanding the adjacent possible. Most of all, it takes the 'genius' pressure off the team. No one knows everything, but each one of us knows a lot about something, and we can all drive plenty of innovation potential by focusing on the spaces we know best. Are you exploring your Adjacent Possible? Share your thoughts below! 👇 #Innovation #innovationsystems #AdjacentPossible

8

2 Comments

I wrote a blog post on a thorny but underappreciated problem in ML-based marketing personalization: event attribution. It's how you go from the actions you take and customer behavior you observe to creating training experiences for your model. Good attribution makes a huge difference in driving uplift, there are a lot of degrees of freedom in configuring it, and at the same time, it's rarely discussed in ML/bandits/RL literature.

47

4 Comments

During yesterday's LibreQoS APNIC webinar, I posed the question: won't more bandwidth solve these problems? As Herbert Wolverson said, yes, more bandwidth is good, but, still these problems remain if queueing is done incorrectly. Here's my take on why bandwidth alone is not as good as bandwidth + good queueing policies: Generally it was believed that 'data is important, so don't throw it away; hang on to it and send it later'. In practice, this has proven to be suboptimal as two issues may emerge: 1) latency increases for some flows due to heavy demand from other flows using the same bottleneck link, 2) even a single flow can have excessive latency due to aspects of typical TCP behavior (referred to bufferbloat) when the buffers grew large enough that the data being buffered was retransmitted anyway. It turns out that not all data is equally important. Active Queue Management is the art of deciding priorities, both in deciding what data to throw away, but also in allowing some later arriving data to be transmitted ahead of data in another connection that arrived before it. These problems have been studied, and good solutions have been found by using certain queueing policies in routers and switches, referred to as “fq_codel’ and “cake”. These track the different flows not by classifying the data, but by watching the behavior. Flows that send relatively little data (DNS lookups), or at a measured pace (video chat) have priority over flows that send a lot of data as quickly as possible (App and System updates, Video and ISO downloads).

10

5 Comments

~40% of Frec’s customers are engineers working in tech, many of whom expect frequent and large capital gains. They also prefer to self-manage their money, which makes Frec an ideal platform. Meet Evan: an engineering leader who upgraded from investing in ETFs to direct indexing. Direct indexing appealed to Evan because it offers comparable returns to his ETFs while saving him money on advisory and ETF fees. “ETFs are true and tested, and giving folks a different way of investing in the market while saving more money is really appealing.” He’s used many other robo-advisors and other fintech tools, but “Frec seems to maximize tax loss harvesting opportunities better than any other product that I’ve used.” He also likes Frec's transparent fee structure.  “I feel like with a lot of other companies, that’s muddled in with the cost of investments,” he told us, “and you don’t really know how much you’re paying or how much you’re not saving as a result. I think Frec does a really good job of laying all of that out and giving you a true understanding of the cost of these investment products.” Read more about his Frec journey here ⬇

45

2 Comments