Skip to content

Keep delayed channel tokens working past the sender's request - #8499

Merged
danlapid merged 1 commit into
mainfrom
dlapid/delayed-channel-token-fallback
Nov 17, 2026
Merged

danlapid merged 1 commit into
mainfrom
dlapid/delayed-channel-token-fallback

Conversation

@danlapid

@danlapid danlapid commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes stubs whose channel token arrives asynchronously, which can break for every use after the first. The recipient then fails each use with pushed external is not a DelayedChannelToken, or with a destroyed-fulfiller error.

When getTokenMaybeSync() returns a promise, as it does for some embedders' Durable Object stubs, the serializer writes a promise capability in place of the token. It later pushes the token to the recipient's ExternalPusher and resolves the promise to the pushed capability. The recipient unwraps that capability, and PromisedTokenizableChannel caches the result, so one failure breaks the stub for good.

Sender. For a stub in a return value, the push ran as an IoContext task and registered no pending event. (#8501 made argument stubs fail the call instead, but return values still go to addTask.) If a stateless entrypoint returned a stub and its request then ended, or hang detection canceled it, the task was dropped and the stub stopped working. A downstream integration test reproduces this: relay-return fails in-process, and with the hang error under process sandboxing. pushDelayedChannelToken() now builds the capability with the new sendDelayedChannelToken(), a promise capability that owns the push, so the push lasts as long as the recipient holds it. The pending external that reports a failed push to the caller now watches that capability, and a pending event is registered until the token is ready.

Recipient. unwrapDelayedChannelToken() required the capability to settle on one of the recipient's own DelayedChannelTokenImpls. getLocalServer() returns null only for a live capability that is not in the set; a broken one throws its own error. So that message means the capability settled on something else: another pusher's token, or a proxy or membrane hook that did not unwrap. I have not pinned down which path does that. DelayedChannelToken gains getToken(), and the recipient uses it when the capability doesn't unwrap locally. This is no less safe than an inline subrequestChannelToken, because the recipient validates the bytes the same way. Unwrapping now copies the token instead of moving it out, so getToken() can still answer afterwards.

Tests:

  • external-pusher-test covers a local unwrap, an unwrap at a pusher the token was not pushed to (this failed before), a delayed token, and a failed token.
  • A downstream integration test passes DO stubs through stateless entrypoints and Durable Objects as arguments and results. Every variant passes with --runs_per_test=5.

🤖 Generated with Claude Code

@danlapid
danlapid force-pushed the dlapid/delayed-channel-token-fallback branch from a0eb093 to 8021ece Compare November 16, 2026 19:24
@codecov-commenter

codecov-commenter commented Nov 16, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.64706% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 66.44%. Comparing base (8b983be) to head (c99a234).

Files with missing lines Patch % Lines
src/workerd/io/external-pusher-test.c++ 89.13% 0 Missing and 5 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8499      +/-   ##
==========================================
+ Coverage   66.42%   66.44%   +0.01%     
==========================================
  Files         794      795       +1     
  Lines      202837   202884      +47     
  Branches    31503    31507       +4     
==========================================
+ Hits       134741   134804      +63     
+ Misses      49644    49627      -17     
- Partials    18452    18453       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

A stub whose channel token is only available asynchronously (a
Durable Object stub in edgeworker, for example) is sent with a promise
capability in place of the token. The sender pushes the token to the
recipient's ExternalPusher once it is ready, and resolves the promise
to the pushed capability, which the recipient unwraps.

Two things could leave the recipient with a stub that fails on every
use:

- For a stub in a return value, the push ran as an IoContext task
  with nothing registered as pending. A stateless entrypoint that
  returned the stub could finish its request, or be canceled by hang
  detection, before the token was ready. That canceled the push and
  broke the capability.
- Unwrapping required the capability to be one of the recipient's own
  DelayedChannelTokenImpls. Anything else failed with "pushed external
  is not a DelayedChannelToken" on every use of the stub.

Now pushDelayedChannelToken() builds the capability with the new
sendDelayedChannelToken(), a promise capability that owns the push
itself, so the push lasts as long as the recipient holds the
capability. The pending external that reports a failed push to the
caller now just watches the capability. A pending event keeps hang
detection away while the token is being produced.

DelayedChannelToken gains a getToken() method, and the recipient falls
back to it when the capability does not unwrap locally. That token is
no more trusted than one sent inline, because the recipient validates
it the same way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@danlapid
danlapid force-pushed the dlapid/delayed-channel-token-fallback branch from 8021ece to c99a234 Compare November 16, 2026 23:52
@danlapid
danlapid requested a review from kentonv November 17, 2026 02:39
@danlapid
danlapid marked this pull request as ready for review November 17, 2026 03:18
@danlapid
danlapid requested review from a team as code owners November 17, 2026 03:18

@clanky-the-clanker clanky-the-clanker Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💬 Approved with comments — No correctness, security or performance problems found. The only notes are a non-blocking test coverage gap on the sender side and a comment-style nit.

Findings: 1 info · 1 suggestion (2 are inline comments on the diff)

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


📋 Code Quality: 💬 1 info · 1 suggestion

The change looks correct; two minor notes on test coverage and comment style.

Recommendation: 💬 Optional Nits

  • [INFO] The sender-side lifecycle fix (push survives the end of the request, pending event, failure reported through pendingExternals) has no test in this repository. (src/workerd/api/worker-rpc.c++:68)
  • [SUGGESTION] Test comment narrates a prior failure instead of stating the property under test. (src/workerd/io/external-pusher-test.c++:43)
View full review
  • Sender-side lifecycle has no test in this repository. external-pusher-test.c++ covers sendDelayedChannelToken() and the getToken() fallback. The main sender fix has no test here: a returned stub that survives the end of the sending request, and the pending event held until the token is ready.
  • A test comment narrates history. The comment at external-pusher-test.c++:43 describes a prior failure. The repository's comment guidelines put that kind of narrative in the commit message.

✅ All other checks passed

🔒 Security: ✅ LGTM

No security concerns. The recipient validates bytes from the getToken() fallback the same way as an inline channel token.


📖 AGENTS.md: ✅ LGTM

No issues found.


⚡ Performance: ✅ LGTM

The extra token copy, the pending event and the fallback RPC run once per stub transfer on small tokens, so their cost is negligible.


📝 Documentation: ✅ LGTM

No issues found.


Reviewed commit: c99a234b

Comment thread src/workerd/api/worker-rpc.c++ Resolved
Comment thread src/workerd/io/external-pusher-test.c++ Resolved
@danlapid
danlapid merged commit e1d8561 into main Nov 17, 2026
24 checks passed
@danlapid
danlapid deleted the dlapid/delayed-channel-token-fallback branch November 17, 2026 20:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants