Casey Clements

Worth reading: How the Apache Arrow Format Accelerates Query Result Transfer by Ian Cook One of the hidden problems of intersystem communication is transcription of data formats. Ian does a wonderful job of explaining the "why" of Apache Arrow as well as the "how" of its advantages. Read on...

8

Cedana enables live migration of stateful workloads including databases and long-running batch analytics. This is important as it enables higher reliability and lower costs (20%-80%). When your spot instance gets revoked, you can feel safe knowing that your workload is resumed on another instance without losing work. Here's a how-to guide to get started running Redis on spot. https://lnkd.in/eNt_hRGX If you're headed to #kubecon let's find time to chat.

53

Long-term memory is "just" a built-in document store that supports `get`, `put` and `search`. But that simplicity is for a reason. We see that memory is still application specific - and so want to give developers low-level building blocks to use In many ways, memory is similar to agents: - Both are highly hyped - Both are hard to define - For both, we see that application specific implementations (e.g. vertical) are winning

171

7 Comments

The shift from traditional ML to LLMs has pushed infrastructure requirements to new limits. At neptune.ai, we’re tackling these challenges by focusing on: → Scalable infrastructure, built to handle high-speed data ingestion and processing. → Real-time data display, adapting seamlessly to complex and large volumes. → Consistent UX quality, even under intense backend demands. — (link to the full interview in the comments) #generativeai #genai #llm

3

1 Comment

Hey everyone, I am an investor and advisor to Katanemo.... They are reinventing the infrastructure primitives for GenAI and launched Arch - https://lnkd.in/gCrbbkEV - an intelligent prompt gateway. Give the project a look on Github, and star the project if it resonates

88

3 Comments

It can be interesting to look back where you came from. Hence: A Tale of Two Curves: Post-Pandemic Retrospective Bond yields were extraordinarily low through most of 2020 and all of 2021. Then, yields surged upward just as extraordinarily over the next nine or ten months. Since then, municipal bond yields have been up and down, but have recently ended up remarkably close to where they were after the 2022 yield run-up, according to ICE yield curves. The story for treasury yields is roughly similar, according to U.S. Department of the Treasury data. In the graphs below, the blue yield curve is from the beginning of 2022. The orange curve is from nine or ten months later in 2022. The green curve is from yesterday. The top graph is for high-grade municipal bonds, and the bottom for US treasury bonds.

10

1 Comment

Brilliant technique by Anthropic. The research continues..I expect more breakthroughs in the retrieval techniques than pure LLM models. At least for AI commercialization in the enterprise space, context matters more than pure generative performance. https://lnkd.in/eRhFXAXP

10

4 Comments

Interesting read by Yingjun Wu of RisingWave about the future of Kafka. https://lnkd.in/eri9-fgX At this point, Kafka is a very important component within almost every modern data infrastructure at a lot of enterprises who are applying Event-Driven Architecture. But how does it need to evolve to keep being it? #kafka #eventdrivenarchitecture #datastreaming

8

Follow up to our "Tiered Storage Won't Fix Kafka" blog post. This one explains all the benefits of WarpStream's Zero Disk Architecture beyond just reducing costs! https://lnkd.in/epjCSVMv

71

1 Comment

How do the costs compare of implementing a low-latency write-ahead-log (WAL) on S3 Express One Zone and one implemented as a State-Machine-Replication (SMR) system (such as Paxos/Raft/Kafka)? I built a cost model for both to find out. https://lnkd.in/dwczVutW

43

5 Comments

In this demonstration, Tom Scott tackled a real-world scenario where a retailer's transaction data was producing erroneous results due to an unusually large exchange rate for the Zimbabwean Dollar. This anomaly caused transactions with this currency to be recorded with a zero value, effectively making them invalid and impacting financial reports and operations. This anomaly caused the transaction amounts to reduce to zero, disrupting operations and creating confusion. By leveraging #indexing, we swiftly identified the root cause of the problem—the incorrect exchange rate—allowing us to rectify the issue promptly. This example highlights how indexing significantly accelerates problem-solving, enabling teams to quickly diagnose and address data anomalies without the need for unnecessary data movement and the complexity of #ETL pipelines. #Kafka #DataStreaming #Indexing #DataAnalytics #BigData #RealTimeData #ETL #DataManagement #StreamProcessing

9

Sad that https://lnkd.in/eAnUjq-i hasn't taken off more -- such a powerful approach. So much energy has gone into the JSONSchema-ification of LLM outputs though, so much less applicable but so much more approachable so it kinda makes sense. For example: https://lnkd.in/e6pW_FNS -- heaven forbid you want an LLM to return both the code and filename to put the code in, you need to take a quality hit to use a structured approach or pray it comes out kinda right if you don't. AICI resolves that super elegantly

5

1 Comment

Databricks has just raised a staggering $10 billion in its latest funding round, achieving a remarkable valuation of $62 billion! This milestone marks one of the largest funding rounds ever and highlights the growing demand for AI-driven solutions. Key takeaways: - Rapid Growth: Databricks is experiencing a 60% year-over-year revenue growth, with projections to reach a revenue run rate of $3 billion by early 2025. - Strategic Investments: The funding will be used to enhance AI product development, expand internationally, and pursue strategic acquisitions. - Market Trends: This round reflects a broader trend in late-stage funding, where investors are favoring established companies with proven growth potential. As the tech landscape evolves, Databricks is positioning itself as a leader in the AI space. This funding not only strengthens its market presence but also sets the stage for exciting innovations ahead! What are your thoughts on this monumental funding round? How do you see it impacting the future of AI and data analytics? 💡👇#Databricks #Funding #AI #TechNews #Investment #Innovation #DataAnalytics

2

📣 Come check out the interview with Sida Shen in Authority Magazine. Sida talks about many of CelerData’s principles, such as: 1. Doing the right thing, not just the easy thing 2. Empathy with users 3. Being curious 4. Always staying humble We deeply understand the significance of data analytics for businesses, which drives us to continuously optimize our products and explore new possibilities. If you’re interested in improving your company’s data analytics capabilities, feel free to reach out to me! 🤝 #dataanalytics #lakehouse #bigdata https://lnkd.in/gEF5tF5E

4

1 Comment

Distributed incremental Spark-style batch jobs: https://lnkd.in/gTh9SD9P Looking for feedback on this, this was a quick prototype, but with a bit of elbow grease it could be used for big data computation with perfect incrementality, no need for manual serialization, no dependency conflicts, and jobs that deploy in seconds with a function call. cc Unison Computing

22

2 Comments

I've been saying this for 2 years now and we are finally here. Deep learning models went through this pruning phase as well. If I recall correctly, they were able to do up to 70% with hardly any loss. SOOOO, what that means is, inference is going to be even cheaper. Open source on-prem models are going to be even more prevalent. And, while pruning has nothing to do with this, it does usually follow, people will start experimenting with new techniques and architecture. Now is the time to keep your eyes peeled. This is where the cool shit starts happening. I give it 6 months before we start hearing about a flurry of new techniques. They are probably already working on the right now, to be honest. Some quiet hidey hole with researchers seeing if they can tease out new designs and new techniques or resurrect old ideas to strengthen the model or give it new capabilities. Maybe it will come from a big company, but they are too focused on products right now, so I don't know how that will fit into the calculus of how things used to work. Definitely keep your ear to ground for the universities or random private research. Updates come from unusual places. Anything that is being worked on right now will start getting finished up here mid next year. Maybe even sooner. My guess, is either completely new architecture but still transformer, or graph based enhancements, mixture of experts improvements, or something using older but solid techniques like q learning, k-means, etc. I suspect a huge reevaluation of ethics training as well. Because that is a cluster fuck that only makes the model worse and encourages it to be non-compliant with instructions. Anyway, long story short, the king is dying, long live the next one.

11

2 Comments

Interesting paper showing that LoRA doesn't learn your data that well which raises questions on how effective it is at fine tuning your Gen AI model. LoRA Learns Less and Forgets Less - https://lnkd.in/eG4tEuSe

19

Snowflake 🤝 Houseware 🤝 LangChain While building AI agents on Snowflake, we realized how the Langchain ecosystem didn't have established connectors to Cortex LLM models. Nipun & Shubham from the team recently built out a Chat Model integration for Snowflake Cortex inside Langchain's community libraries, and raised a pull request just earlier today(link in comments). This abstraction would allows developers to easily build AI applications on top of Cortex LLMs using the Langchain ecosystem, making Cortex a first-class citizen here onwards. OSS FTW 🚀

59

4 Comments

My tldr on the database Stripe recently announced: - Stripe's infra team built DocDB, an internal DBaaS, with its own sharding on top of Mongo (the DB the Collisons chose in 2011), kinda like Vitess on top of MySQL. - > 5M QPS across whole platform, with 5k collections and 2k shards. TBs per shard, petabytes total. - Includes data migration system for better utilization, upgrading Mongo, and autoscaling (future). Migration steps: - Import: take snapshot of data from source shard, insert it into target shard in sorted order to get more proximal writes / fast enough import. - Async replication: writes that happen after the snapshot show up in the source oplog, which get replicated to target via Kafka. - Check: After replication gets in sync (zeroish replication lag), snapshot source & target to check they're the same. - Switch: Source starts returning 5xx, coordinator waits until replication completes, DocDB frontend/proxy service gets the news and starts forwarding subsequent queries to target shard instead of the 5xx-ing source shard. Migration takes less than 2s and all Stripe products retry 5xx error codes for longer than 2s, which means no failed operations during migrations. --- DocDB has 99.999% uptime, or < 26s of downtime per month, which means they can fit ≥ 13 migrations/mo 😄 In reality, I imagine they'll minimize migrations even after implementing autoscaling to try to increase uptime / decrease end-user latency spikes. Would be curious to hear about what operations they support. I don't imagine they've implemented cross-shard transactions (introduced to Mongo in 2019 in v4.2), but it sounds like they've intentionally limited allowed operations for performance reasons. Also curious to hear in what cases the correctness check fails. And to guarantee the < 2s migration, replication lag can't be > 2s at the time of the switch. So before starting the switch, they must confirm that the replication lag hasn't increased too much since starting the correctness check, and either wait for it to go down, or abort and try again later when there's a lower write load on that data? Jimmy Morzaria DocDB post with more details: https://lnkd.in/gdFNVWPg

8

3 Comments