LightSeek Foundation

⚡Meet Qwen3.8-Flash, a multimodal MoE and an early preview of the Qwen4 architecture, now open-weight! The production version Qwen3.8-Flash will be available soon via QwenCloud API at just $ 0.16/1M input tokens and $ 0.47/1M output tokens. 125B parameters + 51B N-gram embeddings, with just 6B activated per token. Unmatched cost-efficiency. What's new: 🔵 Next architecture: GDN + QSA hybrid attention, Gated Residual, N-gram Embedding & Muon optimizer, serving as a precursor to the architecture used in Qwen4. ⚪️ Dramatically lower training and inference costs: trained at just 1/9 the cost of Qwen3.7-Plus, while outperforming it across the board with especially strong gains in coding and office tasks. 🔵 Strong performance: scoring 58.7 on DeepSWE 1.1, 62.5 on SWE-bench Pro, 73.9 on CoWorkBench, 84.5 on AndroidWorld, and 95.7 on MathVision (with CI). ⚪️ 262K native context, extensible to 1M with YaRN. We’re also releasing the weights for Qwen3.8-Flash-Next, giving the community an early look at the new architecture we’re exploring for Qwen4. ✨ We can't wait to see what you build with Qwen3.8-Flash! - Blog: https://lnkd.in/g8CTfwsx - Tech Report：https://lnkd.in/gyU23Rp2 - Hugging Face：https://lnkd.in/guZ9UEje - ModelScope：https://lnkd.in/gGzKchMy

Releasing the model weights and technical report of Kimi K3. Kimi K3 is our most capable model: a 2.8T MoE model with native visual understanding and a 1M-token context window. New model architecture: 2.5x the intelligence per unit of compute, not just more params. Alongside Kimi K3, we're opening up more of the stack behind it — high-performance attention kernels, MoE communication library, and infrastructure for running agent environments at scale. 🌙 Model weights: https://lnkd.in/eZyxBUAZ 🌙 Tech report: https://lnkd.in/giZwywVF 🌙 Tech blog: https://lnkd.in/gnQisRGQ

Today, we are introducing Inkling. Inkling reasons efficiently across text, image, and audio modalities. We are making the full weights available. https://lnkd.in/gY4NvS5h Available today for fine-tuning on Tinker. Play with it in the Inkling Playground. Cost and latency are important in real-world use cases. Inkling's continuous thinking effort lets you pick your point on the cost/performance curve — reaching the same score with a fraction of the tokens. Inkling natively understands and reasons across text, audio, and images. It’s strong on audio in particular, ranking among the strongest open-weights models on VoiceBench, MMAU, and AudioMC. We’re grateful to our partners for their day-0 support across the open-source ecosystem: NVIDIA, Together AI, Fireworks AI, Databricks, Unsloth AI, Modal, Baseten, LightSeek Foundation, Inferact on VLLM, RadixArk on SGLang. Inkling is the first in a family. We’ve included some details of Inkling-Small, a lighter-weight model trained on a similar recipe, with full weights to follow. We hope you enjoy Inkling, and as always we’re keen to see what you build.

One runtime, multiple GPU architectures, and zero vendor-specific model code. In this blog post, the TokenSpeed team LightSeek Foundation introduces TokenSpeed-Kernel, a portable, high-performance kernel system built for modern LLM inference. Using GPT-OSS 120B as a case study, they show how specialized kernels for AMD and NVIDIA GPUs can seamlessly coexist behind a common API. This unified approach delivers up to 3.6x higher throughput on the AMD MI355X, all without requiring any changes to the underlying model logic. Link to blog in comments section 👇

The speed-of-light optimization for Qwen3.5 on the TokenSpeed inference engine is a significant milestone, achieving a record-breaking 580 tokens per second (tps) for agentic workloads on NVIDIA GPUs. In the PyTorch Foundation's latest community blog post, you can learn all about the complete design, implementation, and optimization of Qwen3.5 models in the TokenSpeed inference framework and see for yourself how this work is improving performance 👉 https://lnkd.in/ePSsBVu8 This achievement was a joint effort between the Qwen inference team, LightSeek Foundation, TokenSpeed team, NVIDIA, and the Mooncake team, with special contributions from Tri Dao for FlashAttention-4 (FA4) optimization.

Introducing TokenSpeed, a speed-of-light LLM inference engine. > TensorRT LLM level performance > vLLM level usability > Built by a lean and mission-driven team in two months > MIT license, open-source https://lnkd.in/dfP69kcr https://lnkd.in/dsezEG8V