Skip to content

perf(guardian): Drop assistant text, tool results, and repeated messages from evidence - #3331

Draft
sentry-junior[bot] wants to merge 3 commits into
mainfrom
guardian/compact-evidence
Draft

sentry-junior[bot] wants to merge 3 commits into
mainfrom
guardian/compact-evidence

Conversation

@sentry-junior

@sentry-junior sentry-junior Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Each Guardian review reads the run evidence and the author-labeled thread. The evidence repeated the thread and carried the bulky parts of the run. This cuts it down to what Guardian uses to judge the action.

What Guardian reads now

Before After
User messages In the thread section, and again in the evidence without authors Once, in the thread section. The evidence keeps only messages that the thread section does not show, such as a message whose thread copy was truncated, or any message in a run with no thread section
Assistant text In the evidence (up to the 10k-token message budget, shared with users) Not in the evidence. Replies that people saw stay in the thread section as assistant: lines
Tool calls Name and input Unchanged, same order
Tool results Body A Not shown. marker that keeps the result's place in the order

RUN_RULES and the evidence line in the policy now say this. Dedupe happens when the prompt is rendered (guardianPrompt), so the evals go through it too. It matches on the message text after line-break normalization. A message that the two sides truncate differently is not deduped. It is sent twice, as it was before.

data_exposure and SCRIPT_RULES judge the data in the proposed input and the earlier calls' inputs and status. Neither needs result bodies. A read of a private source followed by a public write still shows up as two calls in order.

Policy changes

  • withheld_step: a condition on the agent's own work, such as tests that pass, is the agent's to check.
  • In a Watch that maintains a pull request, answering a reviewer's question in its thread is part of the work.

Size on the guardian eval fixtures (105 cases)

  • Evidence section: 52,963 → 16,010 characters (−70%)
  • Whole user prompt: 469,735 → 437,544 characters, about 117k → 109k tokens at 4 chars/token (−6.9%)
  • System policy: 13,222 → 13,626 characters, because of the policy changes. With it, the total per call is about the same (+0.6%).

The fixtures already shorten tool output and assistant messages, so these numbers understate the savings in production.

The eval helper still records a full transcript for each case. It builds the thread section from the transcript, then projects the evidence the way core builds it. No case input or expected decision changed.

Prior art

  • Claude Code auto mode: the classifier sees only user messages and tool calls. Assistant text is removed so the agent can't argue its case, and tool results are removed as the main prompt-injection defense. Anthropic says this costs some provenance ("the classifier can't tell whether the agent pulled it from a query or fabricated it"). In their tests, adding assistant messages and tool results made results worse.
  • Codex Guardian reads a compact transcript, and its evidence selection is the base of buildToolActionEvidence.

Not changed

  • Prompt-prefix caching. The system policy, the preamble, and RUN_RULES already come before the random section key. Only SCRIPT_RULES comes after it. Keeping the key the same for a whole run would weaken the section, so I left it alone.
  • Proposal size limits and read-only review are handled in separate PRs.

Open decision

Guardian no longer sees result bodies, so it can't see where some data came from, such as the number of a PR that the agent created. Should we accept that? If not, the fallback is to show a short head of each result.

via @dcramer.

--

View Session [Sentry]

…ges from evidence

Guardian now reads each earlier tool call with its input and only a marker
that its result returned. Assistant text is left out, and user messages that
the author-labeled root section shows are not repeated in the evidence.

Co-Authored-By: David Cramer <23610+dcramer@users.noreply.github.com>
@vercel

vercel Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
junior-docs Ready Ready Preview Nov 13, 2026 5:24am UTC

Request Review

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

11 screenshot changes — 11 changed · 0 added · 0 removed

Review screenshots in Frameshift

Conversation Captured Memories Popover · Desktop
Conversation Captured Memories Popover · Desktop
Changed
Conversation Child Agents · Desktop
Conversation Child Agents · Desktop
Changed
Conversation Details Watched Linked Work · Desktop
Conversation Details Watched Linked Work · Desktop
Changed
Conversation Event Log Virtual · Desktop
Conversation Event Log Virtual · Desktop
Changed
Conversation Event Log Virtual · Mobile
Conversation Event Log Virtual · Mobile
Changed
Conversation Long Transcript Touch · Mobile
Conversation Long Transcript Touch · Mobile
Changed
Show 5 more changed screenshots
Conversation Share Details · Desktop
Conversation Share Details · Desktop
Changed
Conversation Task Line · Desktop
Conversation Task Line · Desktop
Changed
Memories · Mobile
Memories · Mobile
Changed
Review Desk Item · Mobile
Review Desk Item · Mobile
Changed
Settings Mcp · Desktop
Settings Mcp · Desktop
Changed

Without result bodies, Guardian cannot see that tests passed or what an
investigation found. A condition on the outcome of the agent's own work is
the agent's to check, as the policy already says for other checks. A Watch
run answers an event's question where it was asked, as the runtime tells
the agent to.

Co-Authored-By: David Cramer <23610+dcramer@users.noreply.github.com>
Co-Authored-By: David Cramer <23610+dcramer@users.noreply.github.com>

This branch was successfully deployed

1 active deployment
Preview – junior-docs — a487cbad Deployed Nov 13, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant