DigitalOcean

Big thanks to DigitalOcean for their awesome contributions back to the open-source community! Open-sourcing model weights is only half the battle, the other half is building open, reliable serving infrastructure that actually runs them at frontier quality. In their latest engineering deep dive on serving Moonshot AI's 2.78-trillion parameter Kimi K3 model on Day 0, the DO team showcased what real community-driven engineering looks like: 🟣 Leveraging llm-d for GPU Heterogeneity: DigitalOcean built their distributed inference stack using llm-d, taking advantage of its native support for heterogeneous GPU types. This allowed them to seamlessly onboard K3's ~1.56 TB weight footprint across both NVIDIA HGX™ B300 and AMD Instinct™ MI350X platforms without rewriting their core serving layer. 🟣 Collaborative Optimization: They worked alongside open-source maintainers like vLLM to tune the serving recipe, raising batching ceilings, fine-tuning MXFP4 quantization, and speeding up prefill paths. 🟣 Battle-Testing Open Weights: By running K3 through Moonshot's Kimi Vendor Verifier (KVV) suite, they tracked down complex edge cases in dynamic tool calling, schema-constrained decoding, and streaming spec compliance, sharing those fixes and learnings back with the broader ecosystem. Kudos to the DigitalOcean team for proving that serving massive open models on day zero is best done together. 👏 Check out the full "Under the Hood" engineering breakdown: https://lnkd.in/eYXVWavF

Great to see DigitalOcean highlight their use of llm-d to support Kimi K3 support on Day 0. From their guide (link in the comments): "We built our distributed inference stack with llm-d because it includes native support for GPU type heterogeneity. This let us quickly onboard K3 to both AMD and NVIDIA platforms." llm-d's support for multiple accelerators, like Google Cloud TPUs and GPUs from NVIDIA and AMD, make it the best choice for model & AI infrastructure providers. cc: Carlos Costa, Pete Cheslock, Robert Shaw, Abdullah Gharaibeh, Akshay Ram, Nathan Beach, Maroon Ayoub

Getting Kimi (Moonshot AI) K3, all 2.8T parameters of it, production-ready and fast on day zero is complex. We take on that complexity so you don’t have to. 🤖 https://do.co/4w4Saap That meant engineering support for K3's dynamic tools, streaming, and reasoning controls, then verifying our work against Moonshot's own test suite, not our reconstruction of it. Alongside that, we tuned the serving recipe with the vLLM team for NVIDIA HGX™ B300 and AMD Instinct™ MI350X GPUs, so the model isn't just correct, it's fast. The result: you run Kimi K3 on DigitalOcean without thinking about the serving layer at all. This is how. ⬇️

When a senior misses a daily check-in call, someone needs to know immediately. ConfirmOk builds that safety net for police departments, senior communities, and families who can't be there in person. After outages on their old platform put those calls at risk, ConfirmOk moved to DigitalOcean App Platform and Managed PostgreSQL, giving their small team reliable, event-driven infrastructure without a dedicated DevOps hire.

Incredible to see the open model momentum continue. Seeing more than 230 organizations rally around open weights reinforces what’s possible when the ecosystem comes together. Thank you to Microsoft for your partnership, and to everyone helping strengthen the open ecosystem. 🤝

We're looking forward to attending the Berkeley RDI Agentic AI Summit at University of California, Berkeley this weekend! 🗓️📌 https://do.co/4cavn5I Don't miss our session to see how teams are routing between models based on real constraints like cost, latency, task type, preference. We'll close out with a peek at what's on the horizon: evals, caching, and personalization.

You could be overpaying for input tokens. 👀 https://do.co/4wjNgHD Every LLM request reprocesses the same system prompt, tool definitions, and reference docs. Prompt Caching on DigitalOcean AI-Native Cloud automatically reuses that shared context, on supported models, cutting your input cost of that repeated context by up to 80% while reducing latency. In this demo, over 70% of tokens came from cache on the second request.

Kimi (Moonshot AI) K3 is here, and DigitalOcean is proud to be one of vLLM's day-0 inference partners. 🤝 vLLM's engineering makes 1M-token context and native multimodal serving efficient—we make it easy to deploy and scale. Great work, vLLM!

We Built the Same App Twice, With and Without Kimi K3 Go behind the scenes of a real AI application as we build the same app twice: once using a traditional development approach and once with Kimi K3. Compare the results firsthand, then explore the architecture, key design decisions, and performance tradeoffs behind each approach. We'll also share what we learned from deploying both versions in production, including development time, inference costs, latency, and where Kimi K3 accelerated development—or got in the way. Speakers: Austin Black, Senior Solutions Architect II Darian Wilkin, Senior Manager, Solutions Architecture