> This page is for version dev.
> For other versions, use one of these documentation indexes:
> - Latest (v1.6.1) (default): https://docs.nvidia.com/dynamo/latest/llms.txt
> - dev: https://docs.nvidia.com/dynamo/dev/llms.txt
> - v1.6.1: https://docs.nvidia.com/dynamo/v1.6.1/llms.txt
> - v1.6.0: https://docs.nvidia.com/dynamo/v1.6.0/llms.txt
> - v1.5.1: https://docs.nvidia.com/dynamo/v1.5.1/llms.txt
> - v1.5.0: https://docs.nvidia.com/dynamo/v1.5.0/llms.txt
> - v1.4.2: https://docs.nvidia.com/dynamo/v1.4.2/llms.txt
> - v1.4.1: https://docs.nvidia.com/dynamo/v1.4.1/llms.txt
> - v1.4.0: https://docs.nvidia.com/dynamo/v1.4.0/llms.txt
> - v1.3.0: https://docs.nvidia.com/dynamo/v1.3.0/llms.txt
> - v1.2.1: https://docs.nvidia.com/dynamo/v1.2.1/llms.txt
> - v1.2.0: https://docs.nvidia.com/dynamo/v1.2.0/llms.txt
> - v1.1.1: https://docs.nvidia.com/dynamo/v1.1.1/llms.txt
> - v1.1.0: https://docs.nvidia.com/dynamo/v1.1.0/llms.txt
> - v1.0.2: https://docs.nvidia.com/dynamo/v1.0.2/llms.txt
> - v1.0.1: https://docs.nvidia.com/dynamo/v1.0.1/llms.txt
> - v1.0.0: https://docs.nvidia.com/dynamo/v1.0.0/llms.txt
> - v0.9.1: https://docs.nvidia.com/dynamo/v-0-9-1/llms.txt
> - v0.9.0: https://docs.nvidia.com/dynamo/v-0-9-0/llms.txt
> - v0.8.1: https://docs.nvidia.com/dynamo/v-0-8-1/llms.txt
> - v0.8.0: https://docs.nvidia.com/dynamo/v-0-8-0/llms.txt
> - v0.7.1: https://docs.nvidia.com/dynamo/v-0-7-1/llms.txt
> - v0.7.0: https://docs.nvidia.com/dynamo/v-0-7-0/llms.txt

> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Agents

Agent traffic looks different from chat. Each turn repeats a growing prefix, tool calls leave the GPU idle between turns, and some requests (the one a user is waiting on) matter more than others (a background subagent). Dynamo does not run your agent loop. Your harness still owns prompts, tools, subagents, and reasoning state. Dynamo reads a little metadata on each request and uses it to route and schedule better, keep reusable KV cache, and record what happened.

## What You Can Do

| I want to… | Use | Guide |
|---|---|---|
| Point a coding agent at Dynamo | A local endpoint and a harness config | [Run an agent on Dynamo](/dynamo/dev/agents/run-an-agent-on-dynamo) |
| Tell Dynamo which requests belong to which agent | Session IDs | [Identify agent sessions](/dynamo/dev/agents/identify-agent-sessions) |
| Keep a session's turns on the worker that holds its KV cache | Session affinity | [Maintain session affinity](/dynamo/dev/agents/maintain-session-affinity) |
| Let interactive requests go ahead of background work | Priority hints | [Prioritize requests](/dynamo/dev/agents/prioritize-requests) |
| See where time goes between LLM calls and tools | Request tracing | [Trace an Agent Run](/dynamo/dev/agents/agent-tracing) |
| Compare serving configurations without rerunning the agent | Trace replay | [Simulate an Agent Run](/dynamo/dev/agents/agent-simulation) |
| Free KV cache held by agents that are waiting on tools | ThunderAgent | [Schedule Agents with ThunderAgent](/dynamo/dev/agents/thunder-agent-program-scheduler) |

## Try It

Send a request that carries a session ID and a priority hint:

```bash {4,9-11}
curl http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -H 'x-dynamo-session-id: research-run-42:researcher' \
  -d '{
    "model": "my-model",
    "messages": [{"role": "user", "content": "..."}],
    "nvext": {
      "agent_hints": { "priority": 5, "osl": 1024 }
    }
  }'
```

The response is a normal chat completion. The session ID labels the request for tracing. The hints change queueing, scheduling, and cache eviction only under load, and only for the priority layers you turn on. With the launcher defaults they are metadata. See [Prioritize requests](/dynamo/dev/agents/prioritize-requests) for the switches. Neither one pins the session to a worker; for that, see [Maintain session affinity](/dynamo/dev/agents/maintain-session-affinity).

## How Dynamo Uses the Metadata

![The agent harness sends each LLM request with a session ID and optional hints. The Dynamo frontend normalizes the metadata, the router uses it for placement, session binding, and queue order, and the backend engine uses priority for scheduling and eviction. The frontend also records a request trace.](/dynamo/_fern-files/dynamo.docs.buildwithfern.com/ad60fa90736c571230ff312351fe746350b03310fd9157d9b1a65f278232a22c/assets/img/agents-metadata-flow.svg)

| Layer | Signal | What Dynamo does with it |
|---|---|---|
| Frontend API | Session headers and `nvext` request extensions | Normalizes agent identity and serving intent across the Chat Completions, Responses, and Messages APIs. |
| Router | Session ID, priority, expected output length, cache overlap | Places requests for KV reuse, binds sessions to workers when affinity is on, and orders queued work. |
| Backend engine | Priority forwarded by the router | Influences engine scheduling and cache eviction where the backend supports it. |

## Where to Start

1. Follow [Run an agent on Dynamo](/dynamo/dev/agents/run-an-agent-on-dynamo) to get an agent talking to a local deployment.
2. Add [session IDs](/dynamo/dev/agents/identify-agent-sessions) so every request is attributable.
3. Turn on [session affinity](/dynamo/dev/agents/maintain-session-affinity) if your agents run many turns on the same context.
4. Add [priority](/dynamo/dev/agents/prioritize-requests) once interactive and background requests compete for the same workers.
5. Use [tracing](/dynamo/dev/agents/agent-tracing) to measure each change before you keep it.

For SGLang engine flags and subagent KV isolation, see [Agents on SGLang](/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/agents-on-sg-lang).