feat(runtime): trace per-request provider usage and cache attribution - #1277
Merged
Conversation
8 tasks
6 tasks
This was referenced Aug 31, 2026
Astro-Han
added a commit
to Astro-Han/maka-agent
that referenced
this pull request
Sep 4, 2026
Each dispatched provider request was serialized whole and written to the artifact store. The request is built from the conversation the run already holds, so every capture was another copy of the same messages, and each one grew with the conversation: one session here reached 772 MB of captures carrying 4.8 MB of distinct content. The bytes were the smaller cost. Captures were 87% of the artifact population, and every artifact write paid for the whole population, so the capture sink is what turned a growing conversation into quadratic write amplification. Nothing read them. The reader shipped with the capture in apache#1277 and was deleted by apache#2605; the producer stayed. What the panels and diagnostics actually read is the bounded observation on the canonical ModelCallAttempt, which is unchanged. The request is still serialized in memory to size and identify it, and is then dropped. `PreparedRequestMaterial` collapses into the observation it wrapped, and the tracker's per-step capture memo goes with it: its key was the digest, so it never saved the work it appeared to cache. Decoders stay. `captureArtifactId`, the `provider_request_captured` event and the `provider_request_capture` artifact source all still resolve, so attempts and sessions already on disk keep decoding and keep copying. Removing them would fail exactly the records this change is meant to stop producing more of. Tests that used the sink as a hook now use the dispatch gate, and the ones that used it to inspect the outgoing request assert against the provider request bodies instead — the stronger evidence of the two. Closes apache#4082 Generated-by: Claude Code
Astro-Han
added a commit
to Astro-Han/maka-agent
that referenced
this pull request
Sep 4, 2026
Each dispatched provider request was serialized whole and written to the artifact store. The request is built from the conversation the run already holds, so every capture was another copy of the same messages, and each one grew with the conversation: one session here reached 772 MB of captures carrying 4.8 MB of distinct content. The bytes were the smaller cost. Captures were 87% of the artifact population, and every artifact write paid for the whole population, so the capture sink is what turned a growing conversation into quadratic write amplification. Nothing read them. The reader shipped with the capture in apache#1277 and was deleted by apache#2605; the producer stayed. What the panels and diagnostics actually read is the bounded observation on the canonical ModelCallAttempt, which is unchanged. The request is still serialized in memory to size and identify it, and is then dropped. `PreparedRequestMaterial` collapses into the observation it wrapped, and the tracker's per-step capture memo goes with it: its key was the digest, so it never saved the work it appeared to cache. Decoders stay. `captureArtifactId`, the `provider_request_captured` event and the `provider_request_capture` artifact source all still resolve, so attempts and sessions already on disk keep decoding and keep copying. Removing them would fail exactly the records this change is meant to stop producing more of. Tests that used the sink as a hook now use the dispatch gate, and the ones that used it to inspect the outgoing request assert against the provider request bodies instead — the stronger evidence of the two. Closes apache#4082 Generated-by: Claude Code
Astro-Han
added a commit
to Astro-Han/maka-agent
that referenced
this pull request
Sep 4, 2026
Each dispatched provider request was serialized whole and written to the artifact store. The request is built from the conversation the run already holds, so every capture was another copy of the same messages, and each one grew with the conversation: one session here reached 772 MB of captures carrying 4.8 MB of distinct content. The bytes were the smaller cost. Captures were 87% of the artifact population, and every artifact write paid for the whole population, so the capture sink is what turned a growing conversation into quadratic write amplification. Nothing read them. The reader shipped with the capture in apache#1277 and was deleted by apache#2605; the producer stayed. What the panels and diagnostics actually read is the bounded observation on the canonical ModelCallAttempt, which is unchanged. The request is still serialized in memory to size and identify it, and is then dropped. `PreparedRequestMaterial` collapses into the observation it wrapped, and the tracker's per-step capture memo goes with it: its key was the digest, so it never saved the work it appeared to cache. Decoders stay. `captureArtifactId`, the `provider_request_captured` event and the `provider_request_capture` artifact source all still resolve, so attempts and sessions already on disk keep decoding and keep copying. Removing them would fail exactly the records this change is meant to stop producing more of. Tests that used the sink as a hook now use the dispatch gate, and the ones that used it to inspect the outgoing request assert against the provider request bodies instead — the stronger evidence of the two. Closes apache#4082 Generated-by: Claude Code
Astro-Han
added a commit
that referenced
this pull request
Sep 4, 2026
Every model call stored a copy of the conversation. The prepared provider request was serialized whole into a private Artifact, and the record beside it carried up to 256 per-segment rows -- so one call cost tens of KB of database plus a file that grew with the Session it belonged to. Nothing read either: the capture's reader shipped in #1277 and was deleted in #2605, which kept the producer, and the per-segment detail's only consumer folded it into four byte totals. Both producers are gone. The fold now runs where the request is dispatched, and its result -- four byte totals plus a capped tool list, 1,971 B flat -- lands on the canonical ModelCallAttempt, which already owns the request's facts. Per model call with 60 tools and a 200-message conversation: database rows 46,077 to 1,971 B, capture file 136,967 B to none. The Artifact store also sealed one snapshot per Session on every load and every mutation, and a snapshot's revision hashes every record in its Session -- so 400 Sessions were sorted and hashed to answer a question about one. Every reader reloads the whole store from the database first, so a kept snapshot never survived to be read. Sealing one when a reader asks for it deletes the map, the two methods that maintained it, and the per-mutation bookkeeping: one listPage at 6,000 records goes 13.45 to 11.54 ms. Compatibility: every decoder stays. `hasExactShape` fails a whole record on an unknown key, so removing `captureArtifactId`, the `provider_request_capture` source, or `PreparedRequestObservation` and its validator would strand exactly the records this stops producing more of -- including their usage and cost. Captures already written are left on disk; #4738 reclaims them. One model-visible change: a sub-agent's spawn tool result listed the private capture in `artifactIds` / `artifactCount`. A child turn now stores nothing of its own, so that list is empty. Also gives `graceful Host shutdown stops and drains an active Turn` the checkpoint its sibling test already used, so the state its drain finds is not left to how fast the machine is. Closes #4082 Refs #4037 Refs #4704 Generated-by: Claude Code
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
events.jsonl, without a provider proxy or second telemetry file.Closes #1268
Verification
npm run format:checknpm run lintnpm run buildnpm run test:fast— all workspace tests passedSecurity
Prepared request bodies omit headers and abort signals, live only in artifacts marked
provider_request_capture, and are filtered from the desktop artifact pane. AgentRun and RuntimeEvent records contain only hashes, sizes, segment metadata, artifact references, and the aggregate trace id.