University of Utah

Teaching Agents in Their Own Style: Black-Box Teacher Distillation for Web Agents

Kahlert School of Computing · University of Utah

ANATOMY OF A SMALL CORRECTION

The student’s task

Find the price of the most expensive bookshelf.

BEFOREfill('15', 'bookshelf')
AFTERfill('31', 'bookshelf')
Right idea.
  Wrong element.
FIG. 01   /   ILLUSTRATIVE CORRECTION, ADAPTED FROM THE PAPER

Teaching Agents in Their Own Style:
Black-Box Teacher Distillation for Web Agents

Style-preserving teacher distillation

WEBKD trains web agents on text-only corrections from a black-box teacher. The teacher edits a failed trajectory from its first error onward, fixing decisions while keeping the student’s own reasoning style.

Illustrative example adapted from Figure 1 of the paper; text simplified for illustration.
Observation[15] link ‘Simple Shop’[31] textbox ‘Search’[47] button ‘Search’
ThinkEdited

I need to search for “bookshelf”. The search box is 31, so I’ll type there.

Plan

1. Type bookshelf → 2. Search → 3. Compare prices.

ActionEditedfill('31', 'bookshelf')

The action is corrected, while the short, direct phrasing stays close to the student’s style.

Method

The student learns from teacher edits to its own failed attempts, with more weight on the reasoning that supports each corrected action.

Speed 1×
Read as text
  1. A behavior-cloned student makes a browser attempt, shown as four original contexts and their responses.
  2. The teacher identifies the first erroneous step. Responses before this point are kept verbatim.
  3. Each response from the error onward is independently corrected at its original student context, in the student’s style. Edited actions do not generate a new teacher trajectory.
  4. The perplexity premium measures extra mean per-token surprisal under the frozen student. In the schematic comparison, original student text has a short bar, a raw teacher edit has a long bar, and a style-conditioned edit has a short bar. Bar length, not color, represents surprisal.
  5. For each reasoning span, the frozen seed scores the corrected action with and without that span. The difference is its action contribution.
  6. More informative spans receive more training weight; less informative spans are downweighted, not discarded. When a step’s contributions are nearly tied, their weights move toward a shared midpoint. Corrected action tokens retain full imitation weight.
  7. The objective adds a weighted mean of cross-entropies to a normalized pointwise KL surrogate. The role mask is 1 on think, plan, and memory, and 10⁻⁶ on action and step.
  8. The anchor keeps reasoning close to the seed without freezing it. The action is almost unanchored so the student can adopt corrected decisions.
Static diagram
  1. Behavior cloning

    Behavior-clone successful teacher demonstrations to give the student its initial web skills.

  2. Student rollouts

    Collect its own trajectories. Failed attempts become edit targets; successes provide examples of its style.

  3. Error localization

    The teacher identifies the earliest error that sends the task off course. Earlier correct steps stay intact.

  4. Style-preserving edits

    Correct each response in the faulty suffix at the student’s original context, using its writing as a guide.

Training objective and equations
Eq. 5

Weighted imitation and rationale anchoring

L WD equals the sum of w i times CE i divided by the sum of w i, plus beta times the sum of gamma i times kl i divided by the sum of gamma i.
Weighted imitation
Emphasize reasoning spans that support the corrected action. Action and step tokens receive full imitation weight.
Rationale anchor
Limit drift from the frozen seed on reasoning tokens. Action and step tokens are almost unanchored.

Each term is normalized by its own weight sum; β controls the anchor strength.

Eq. 2

Perplexity premium

The premium is teacher-supervision mean surprisal minus student-response mean surprisal under the frozen seed.

This is a difference in log perplexities, not raw perplexities. Uniform training can spend its largest loss on unfamiliar wording instead of the useful decision.

Eq. 4

Leave-one-out action contribution

Delta s is the frozen student's log probability of the corrected action with the span minus without the span.

Both scores use the frozen behavior-cloned student. A span earns credit when its presence increases the probability of the corrected action. Negative contributions stay in the ranking.

Eq. 10

Within-step rank weights

Rank-derived weights are contracted toward the midpoint according to contribution spread, then clipped between phi and one.

Ranks are normalized within each step; ties share their average rank. With φ = 0.1 and τ = 0.05 nats, identical contributions yield 0.55. Action and step CE weights are pinned to 1.

Eq. 5

Role-asymmetric anchoring

The pointwise penalty is exp d minus d minus one; d is the seed log probability minus the current policy log probability of a target token.

Here pᵢ is the current student’s target-token probability and qᵢ is the frozen seed’s. On corpus target tokens, use γ = 1 for think/plan/memory and γ = 10⁻⁶ elsewhere. This pointwise surrogate is not an exact full-vocabulary KL. β = 0.03 in the reported configuration.

Notation: πBC is the frozen behavior-cloned seed, πθ is the student being trained. Appendix B.4–B.5

Results

Task success rate on TimeWarp and on live websites with OnlineMind2Web. Each bar is the mean of three runs; whiskers show ±1 standard error.

TimeWarp v6 (seen)

TimeWarp v6

See Table 1 of the preprint for the reported values.

Success rate (%)

Selected baselines WEBKD ±1 standard error

TimeWarp v1–v5 (unseen)

TimeWarp v1

See Table 1 of the preprint for the reported values.

Success rate (%)

Selected baselines WEBKD ±1 standard error

OnlineMind2Web

OnlineMind2Web · Overall

See Figure 4 of the preprint for the reported values.

Success rate (%)

Baselines WEBKD ±1 standard error

Cite this work

Md Farhan Ishmam and Kenneth Marino. Teaching Agents in Their Own Style: Black-Box Teacher Distillation for Web Agents. arXiv:2610.19834, 2026.

Read on arXiv
BibTeX
@misc{ishmam2026webkd,
  title={Teaching Agents in their Own Style:
    Blackbox Teacher Distillation for Web Agents},
  author={Ishmam, Md Farhan and Marino, Kenneth},
  year={2026},
  eprint={2610.19834},
  archivePrefix={arXiv},
  primaryClass={cs.AI},
  url={https://arxiv.org/abs/2610.19834}
}