This page is for version dev.
For other versions, use one of these documentation indexes:
- Latest (v1.6.1) (default): https://docs.nvidia.com/dynamo/latest/llms.txt
- dev: https://docs.nvidia.com/dynamo/dev/llms.txt
- v1.6.1: https://docs.nvidia.com/dynamo/v1.6.1/llms.txt
- v1.6.0: https://docs.nvidia.com/dynamo/v1.6.0/llms.txt
- v1.5.1: https://docs.nvidia.com/dynamo/v1.5.1/llms.txt
- v1.5.0: https://docs.nvidia.com/dynamo/v1.5.0/llms.txt
- v1.4.2: https://docs.nvidia.com/dynamo/v1.4.2/llms.txt
- v1.4.1: https://docs.nvidia.com/dynamo/v1.4.1/llms.txt
- v1.4.0: https://docs.nvidia.com/dynamo/v1.4.0/llms.txt
- v1.3.0: https://docs.nvidia.com/dynamo/v1.3.0/llms.txt
- v1.2.1: https://docs.nvidia.com/dynamo/v1.2.1/llms.txt
- v1.2.0: https://docs.nvidia.com/dynamo/v1.2.0/llms.txt
- v1.1.1: https://docs.nvidia.com/dynamo/v1.1.1/llms.txt
- v1.1.0: https://docs.nvidia.com/dynamo/v1.1.0/llms.txt
- v1.0.2: https://docs.nvidia.com/dynamo/v1.0.2/llms.txt
- v1.0.1: https://docs.nvidia.com/dynamo/v1.0.1/llms.txt
- v1.0.0: https://docs.nvidia.com/dynamo/v1.0.0/llms.txt
- v0.9.1: https://docs.nvidia.com/dynamo/v-0-9-1/llms.txt
- v0.9.0: https://docs.nvidia.com/dynamo/v-0-9-0/llms.txt
- v0.8.1: https://docs.nvidia.com/dynamo/v-0-8-1/llms.txt
- v0.8.0: https://docs.nvidia.com/dynamo/v-0-8-0/llms.txt
- v0.7.1: https://docs.nvidia.com/dynamo/v-0-7-1/llms.txt
- v0.7.0: https://docs.nvidia.com/dynamo/v-0-7-0/llms.txt
Dynamo publishes 25 Rust crates for release 1.6.0.
dynamo-kv-router
KV-aware request routing library
dynamo-llm
LLM inference engine
dynamo-memory
Memory management utilities
dynamo-runtime
Core distributed runtime library
kvbm-logical
Logical layer for the KV Block Manager
dynamo-bench
Lightweight HTTP benchmarks for Dynamo endpoints
dynamo-config
Configuration management
dynamo-data-gen
Schemas and primitives for Dynamo data generation and replay traces
dynamo-kv-hashing
Request-to-lineage-hash contract for KV cache identity
dynamo-parsers
Protocol parsers (SSE, JSON streaming)
dynamo-parsers-v2
Successor parser line to dynamo-parsers, consumed by the Frontend
dynamo-protocols
Async OpenAI-compatible API client
dynamo-renderer
Chat-template rendering used by the Dynamo Frontend
dynamo-rl
Dynamo RL worker discovery API
dynamo-tokenizers
Tokenizer library for LLM inference
dynamo-tokens
Tokenizer bindings for LLM inference
dynamo-truthy
Canonical truthy/falsy boolean flag parsing
fastokens
Rust BPE tokenizer backend consumed by the Frontend
kvbm-common
Shared types for the KV Block Manager
kvbm-config
KVBM configuration for Tokio, Rayon, and Messenger runtimes
kvbm-engine
Distributed coordination primitives for KVBM
kvbm-kernels
CUDA kernels for the KV Block Manager
kvbm-physical
Physical block layer for the KV Block Manager
dynamo-mocker
Inference engine simulator for benchmarking
dynamo-async-openai
Legacy OpenAI client; use dynamo-protocols
Core Crates
| Crate | Summary | Version | Install |
|---|---|---|---|
dynamo-kv-router | KV-aware request routing library | 1.6.0 | cargo add dynamo-kv-router@1.6.0 |
dynamo-llm | LLM inference engine | 1.6.0 | cargo add dynamo-llm@1.6.0 |
dynamo-memory | Memory management utilities | 1.6.0 | cargo add dynamo-memory@1.6.0 |
dynamo-runtime | Core distributed runtime library | 1.6.0 | cargo add dynamo-runtime@1.6.0 |
kvbm-logical | Logical layer for the KV Block Manager | 1.6.0 | cargo add kvbm-logical@1.6.0 |
Supporting Crates
| Crate | Summary | Version | Install |
|---|---|---|---|
dynamo-bench | Lightweight HTTP benchmarks for Dynamo endpoints | 1.6.0 | cargo add dynamo-bench@1.6.0 |
dynamo-config | Configuration management | 1.2.1 | cargo add dynamo-config@1.2.1 |
dynamo-data-gen | Schemas and primitives for Dynamo data generation and replay traces | 1.6.0 | cargo add dynamo-data-gen@1.6.0 |
dynamo-kv-hashing | Request-to-lineage-hash contract for KV cache identity | 1.6.0 | cargo add dynamo-kv-hashing@1.6.0 |
dynamo-parsers | Protocol parsers (SSE, JSON streaming) | 7.0.1 | cargo add dynamo-parsers@7.0.1 |
dynamo-parsers-v2 | Successor parser line to dynamo-parsers, consumed by the Frontend | 0.1.23 | cargo add dynamo-parsers-v2@0.1.23 |
dynamo-protocols | Async OpenAI-compatible API client | 5.4.1 | cargo add dynamo-protocols@5.4.1 |
dynamo-renderer | Chat-template rendering used by the Dynamo Frontend | 5.1.2 | cargo add dynamo-renderer@5.1.2 |
dynamo-rl | Dynamo RL worker discovery API | 1.6.0 | cargo add dynamo-rl@1.6.0 |
dynamo-tokenizers | Tokenizer library for LLM inference | 1.8.1 | cargo add dynamo-tokenizers@1.8.1 |
dynamo-tokens | Tokenizer bindings for LLM inference | 1.6.0 | cargo add dynamo-tokens@1.6.0 |
dynamo-truthy | Canonical truthy/falsy boolean flag parsing | 1.6.0 | cargo add dynamo-truthy@1.6.0 |
fastokens | Rust BPE tokenizer backend consumed by the Frontend | 0.2.0 | cargo add fastokens@0.2.0 |
kvbm-common | Shared types for the KV Block Manager | 1.6.0 | cargo add kvbm-common@1.6.0 |
kvbm-config | KVBM configuration for Tokio, Rayon, and Messenger runtimes | 1.6.0 | cargo add kvbm-config@1.6.0 |
kvbm-engine | Distributed coordination primitives for KVBM | 1.6.0 | cargo add kvbm-engine@1.6.0 |
kvbm-kernels | CUDA kernels for the KV Block Manager | 1.6.0 | cargo add kvbm-kernels@1.6.0 |
kvbm-physical | Physical block layer for the KV Block Manager | 1.6.0 | cargo add kvbm-physical@1.6.0 |
Development and Testing
| Crate | Summary | Version | Install |
|---|---|---|---|
dynamo-mocker | Inference engine simulator for benchmarking | 1.6.0 | cargo add dynamo-mocker@1.6.0 |
Deprecated Crates
| Crate | Summary | Version | Install |
|---|---|---|---|
dynamo-async-openai | Legacy OpenAI client; use dynamo-protocols | 1.0.2 | cargo add dynamo-async-openai@1.0.2 |
Language Bindings
| Binding | Language | Summary | Source |
|---|---|---|---|
dynamo-codegen | Python | PyO3 code generation for the dynamo._core module. | lib/bindings/python/codegen |
libdynamo_llm | C | C bindings for the Dynamo LLM library. | lib/bindings/c |