Skip to content

Let container port pumps reach the capnp streams - #8273

Merged
gabivlj merged 2 commits into
mainfrom
gv/container-port-pump-v2
Nov 10, 2026
Merged

gabivlj merged 2 commits into
mainfrom
gv/container-port-pump-v2

Conversation

@gabivlj

@gabivlj gabivlj commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

Pumps through a container port, such as a fetch() request body or a connect() socket, were slower
than they need to be, because ContainerPortAsyncIoStream did not override pumpTo() or
tryPumpFrom(). A pump from the port read down in a generic loop, 4 KiB at a time with an extra
copy, or 8 KiB into each RPC message when the destination was a capnp ByteStream. A pump into the
port used the source's fallback, often the same 4 KiB loop. Neither direction could redirect a
capnp ByteStream to the other side with getSubstream().

pumpTo() now delegates to the down pipe. Each write the container makes on down goes straight
to the destination, and when the destination is a capnp ByteStream, down can be redirected to it
with getSubstream().

tryPumpFrom() now delegates to the up ByteStream once Port.connect() has returned. The adapter
reads the source straight into RPC messages or, when the source is itself a capnp ByteStream,
redirects it to up with getSubstream().

fetch() and connect() still fail with Port.connect()'s error when the tunnel fails to open:

  • Until Port.connect() has returned, tryPumpFrom() returns none and the caller falls back to
    write(), as before. If the tunnel fails to open, a redirected source's getSubstream() fails and
    the source's pump is cancelled with a generic error that does not pass through this stream, so
    it could arrive ahead of Port.connect()'s error. Pumps that start before Port.connect() returns,
    such as the first request body on a new tunnel, are therefore written as before; a request body
    on a reused tunnel can be redirected.
  • A failed pump is handled as an unclean disconnect and rejects with the stream's disconnect
    exception. While Port.connect() is outstanding this waits for its result, like the existing
    disconnect signals, so a pump broken by the tunnel failing to open reports Port.connect()'s
    error. notifyUncleanDisconnect() now keeps the exception of the first such signal, which is used
    if Port.connect() succeeds.

Other behaviour changes:

  • Every failed pump now tears down the tunnel, including failures caused by the other stream, such
    as a request body that errors or a destination that rejects writes. The connection cannot be
    reused after a partial pump anyway.
  • If the sender of a body that is redirected to up aborts, the caller sees capnp's "stream
    disconnected because SubstreamCallbackImpl was never called back" instead of the body stream's
    own error, and that becomes the tunnel's disconnect exception. The pooled HTTP path can hit this
    when a client aborts an upload.
  • Cancelling a redirected pump does not revoke the substream, so the sender can keep writing into
    up after this side has given up on the connection. This is inherent to redirecting with
    getSubstream(); the write-based pump did not do it.
  • When a raw connect()'s down is redirected, for example because the Worker pumps the socket's
    readable into a stream sent over RPC, an unclean end of the tunnel fails that pump with a
    generic error. Before, the readable ended, because the HTTP client adapter shuts the socket's
    pipe down when connect() fails after the socket was accepted.

The container mock now reads request bodies, so tests can check what reached up. The new tests
pump a request body while Port.connect() is outstanding and then fails, with getSubstream()
failing with a different error; pump one on a reused tunnel, where it is redirected to up and
arrives intact; pump a raw connect()'s socket into a capnp ByteStream, to which down is
redirected; and drop down after that redirect before Port.connect() fails.

@gabivlj
gabivlj requested review from a team as code owners November 7, 2026 00:28

@clanky-the-clanker clanky-the-clanker Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by the review of b33bdaf1.

This review, of e1187014

💬 Approved with comments — The pump delegation and the disconnect-exception bookkeeping look correct. One WARNING: a pump that fails while Port.connect() is outstanding now waits for connect() to settle, and that wait can stall if connect() stays outstanding for the tunnel's lifetime.

Findings: 1 warning (1 is an inline comment on the diff)

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.

🔧 Fix with your agent

Paste the following into your coding agent to investigate and fix these issues.

Fix the following code review issues found in pull request #8273.

Before making changes, review each issue and present a brief summary table:
- For each finding, state whether you agree, disagree, or need clarification
- If you disagree (e.g., the fix would require disproportionate changes
  for minimal ROI, or the finding is factually incorrect), explain why
- If you need to clarify something before deciding, ask those questions
- Then share your plan for which issues to tackle and in what order

## WARNING (1)

### [Code Quality] A pump failure while Port.connect() is outstanding is held until connect() settles. It can stall if connect() stays outstanding for the tunnel's lifetime, which the removed test comment described.
- **File:** src/workerd/api/container.c++:1207

📋 Code Quality: ⚠️ 1 warning

The pump delegation is sound. A pump failure during the connecting window waits on Port.connect(), which could stall depending on when the real service returns.

Recommendation: 📝 Minor Comments

  • [WARNING] A pump failure while Port.connect() is outstanding is held until connect() settles. It can stall if connect() stays outstanding for the tunnel's lifetime, which the removed test comment described. (src/workerd/api/container.c++:1207)
View full review

The new pumpTo() and tryPumpFrom() overrides delegate correctly to down and up. notifyUncleanDisconnect() keeps only the first exception it sees, as intended.

A pump failure while connect() is outstanding waits for connect() (src/workerd/api/container.c++:1202-1210)

  • propagatePumpFailure() handles every pump failure as an unclean disconnect and returns rejectWhenDisconnected(). While connectState is PENDING, that promise rejects only after Port.connect() settles.
  • Take a fresh connection where the destination of pumpTo() breaks, for example a client that went away. That failure used to reach the caller at once. Now it is held until connect() returns.
  • This pull request rewrites the test comment at container-test.c++:1350. The old comment said the real container service keeps Port.connect() outstanding for the lifetime of the connection. The new one says it returns once the container accepts the connection.
  • If the old description is right, two things follow:
    • The held pump failure is released only when the tunnel is torn down, and the caller waiting on the pump may be what keeps the tunnel open. That can stall.
    • tryPumpFrom() never takes the getSubstream() path on a fresh connection.
  • The tests cover two cases: a connect() that fails while a pump is outstanding, and a reused tunnel. None covers a pump failure while connect() is outstanding that is followed by a successful connect().
  • Please confirm when the real service returns from Port.connect(). If it can stay outstanding after the tunnel is usable, one option is to propagate failures that did not come from down or up right away.

✅ All other checks passed

🔒 Security: ✅ LGTM

No security issues found.


📖 AGENTS.md: ✅ LGTM

No AGENTS.md issues found.


⚡ Performance: ✅ LGTM

No performance issues found.


📝 Documentation: ✅ LGTM

No documentation issues found.


Reviewed commit: e1187014

Comment thread src/workerd/api/container.c++ Resolved

@danlapid danlapid left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The state handling looks right to me, and I couldn't find a correctness bug. My comments are mostly about what the tests prove, plus two descriptions that don't match production.

I checked the production side, since the up redirect depends on it. cloudchamber's byteStreamWrite (cc/cloudchamber/go/pkg/capnp/bytestream.go) implements getSubstream(), including limits, ended/reachedLimit and nested substreams. portCapnp.Connect returns only after the container has accepted the connection. So the connectState == SETTLED check only blocks the redirect for the first request on a new tunnel, and requests that reuse a pooled tunnel get it.

One correction to the PR description: it says that after down is redirected, "the container's bytes no longer pass through this process". C++ capnp has no three-party handoff, so the container's writes still reach this process and the RPC layer forwards them to the destination. What's saved is the extra copy and the 4 KiB read/write loop, not the hop through this process. It's worth rewording so nobody relies on the stronger claim.

Comment thread src/workerd/api/container-test.c++ Resolved
Comment thread src/workerd/api/container-test.c++ Resolved
Comment thread src/workerd/api/container-test.c++ Resolved
Comment thread src/workerd/api/container.c++ Resolved
Comment thread src/workerd/api/container.c++ Resolved
Comment thread src/workerd/api/container.c++ Resolved
@gabivlj
gabivlj force-pushed the gv/container-port-pump-v2 branch from e118701 to b33bdaf Compare November 9, 2026 17:02

@clanky-the-clanker clanky-the-clanker Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by the review of 828e94a5.

This review, of b33bdaf1

✅ Approved — No open findings. The earlier warning about a pump failure waiting for Port.connect() is resolved: the discussion confirms that connect() returns once the container accepts the connection, and the code matches that.

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


✅ All checks passed

🔒 Security: ✅ LGTM

No security issues found.


📖 AGENTS.md: ✅ LGTM

No AGENTS.md issues found.


📋 Code Quality: ✅ LGTM

The pump overrides delegate correctly, and the disconnect-exception bookkeeping holds up. The earlier concern about a pump failure being held until Port.connect() settles no longer applies: the service returns connect() once the container accepts the connection, as confirmed in the discussion, and the existing disconnect signals already wait in the same way. The new tests cover both the up and down redirect paths and check that the body arrives.


⚡ Performance: ✅ LGTM

The change removes the extra buffered copy on container port pumps. The failure-handling path adds nothing to successful pumps.


📝 Documentation: ✅ LGTM

No documentation issues found.


Reviewed commit: b33bdaf1

@gabivlj
gabivlj requested a review from danlapid November 9, 2026 17:52
Comment thread src/workerd/api/container-test.c++ Resolved
@codecov-commenter

codecov-commenter commented Nov 9, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 82.84672% with 47 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.48%. Comparing base (9d4e0f5) to head (7e866a4).

Files with missing lines Patch % Lines
src/workerd/api/container-test.c++ 83.40% 19 Missing and 21 partials ⚠️
src/workerd/api/container.c++ 78.78% 4 Missing and 3 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8273      +/-   ##
==========================================
+ Coverage   65.46%   65.48%   +0.02%     
==========================================
  Files         759      759              
  Lines      192409   192659     +250     
  Branches    30344    30375      +31     
==========================================
+ Hits       125951   126158     +207     
- Misses      48841    48861      +20     
- Partials    17617    17640      +23     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Pumps through a container port, such as a fetch() request body or a connect() socket, were slower
than they need to be, because ContainerPortAsyncIoStream did not override pumpTo() or
tryPumpFrom(). A pump from the port read `down` in a generic loop, 4 KiB at a time with an extra
copy, or 8 KiB into each RPC message when the destination was a capnp ByteStream. A pump into the
port used the source's fallback, often the same 4 KiB loop. Neither direction could redirect a
capnp ByteStream to the other side with getSubstream().

pumpTo() now delegates to the `down` pipe. Each write the container makes on `down` goes straight
to the destination, and when the destination is a capnp ByteStream, `down` can be redirected to it
with getSubstream().

tryPumpFrom() now delegates to the `up` ByteStream once Port.connect() has returned. The adapter
reads the source straight into RPC messages or, when the source is itself a capnp ByteStream,
redirects it to `up` with getSubstream().

C++ capnp has no three-party handoff, so redirected bytes still reach this process, and the RPC
layer forwards them to the other stream. A redirect saves the copy and the read/write loop, not the
hop through this process.

fetch() and connect() still fail with Port.connect()'s error when the tunnel fails to open:

- Until Port.connect() has returned, tryPumpFrom() returns none and the caller falls back to
  write(), as before. If the tunnel fails to open, a redirected source's getSubstream() fails and
  the source's pump is cancelled with a generic error that does not pass through this stream, so
  it could arrive ahead of Port.connect()'s error. Pumps that start before Port.connect() returns,
  such as the first request body on a new tunnel, are therefore written as before; a request body
  on a reused tunnel can be redirected.
- A failed pump is handled as an unclean disconnect and rejects with the stream's disconnect
  exception. While Port.connect() is outstanding this waits for its result, like the existing
  disconnect signals, so a pump broken by the tunnel failing to open reports Port.connect()'s
  error. notifyUncleanDisconnect() now keeps the exception of the first such signal, which is used
  if Port.connect() succeeds.

Other behaviour changes:

- Every failed pump now tears down the tunnel, including failures caused by the other stream, such
  as a request body that errors or a destination that rejects writes. The connection cannot be
  reused after a partial pump anyway.
- If the sender of a body that is redirected to `up` aborts, the caller sees capnp's "stream
  disconnected because SubstreamCallbackImpl was never called back" instead of the body stream's
  own error, and that becomes the tunnel's disconnect exception. The pooled HTTP path can hit this
  when a client aborts an upload.
- Cancelling a redirected pump does not revoke the substream, so the sender can keep writing into
  `up` after this side has given up on the connection. This is inherent to redirecting with
  getSubstream(); the write-based pump did not do it.
- When a raw connect()'s `down` is redirected, for example because the Worker pumps the socket's
  readable into a stream sent over RPC, an unclean end of the tunnel fails that pump with a
  generic error. Before, the readable ended, because the HTTP client adapter shuts the socket's
  pipe down when connect() fails after the socket was accepted.

The container mock now reads request bodies, so tests can check what reached `up`. The new tests
pump a request body while Port.connect() is outstanding and then fails, with getSubstream()
failing with a different error; pump one on a reused tunnel, where it is redirected to `up` and
arrives intact; pump a raw connect()'s socket into a capnp ByteStream, to which `down` is
redirected; and drop `down` after that redirect before Port.connect() fails.
@gabivlj
gabivlj force-pushed the gv/container-port-pump-v2 branch from b33bdaf to 828e94a Compare November 9, 2026 19:33

@clanky-the-clanker clanky-the-clanker Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by the review of 7e866a47.

This review, of 828e94a5

✅ Approved — No open findings. The test changes since the last review fix the coverage gaps raised in the discussion, and the production pump logic still holds up.

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


✅ All checks passed

🔒 Security: ✅ LGTM

No security issues found.


📖 AGENTS.md: ✅ LGTM

No AGENTS.md issues found.


📋 Code Quality: ✅ LGTM

pumpTo() and tryPumpFrom() delegate correctly. In each connect state, a failed pump rejects with the stored disconnect exception. The updated tests use a distinct getSubstream() error, check that the body reached up, implement tryReadSync() on the mock input, and cover the down redirect.


⚡ Performance: ✅ LGTM

Container port pumps no longer make the extra buffered copy, and the failure handling adds no work to successful pumps.


📝 Documentation: ✅ LGTM

No documentation issues found.


Reviewed commit: 828e94a5

@gabivlj
gabivlj requested a review from jasnell November 9, 2026 19:48

@danlapid danlapid left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this addresses everything I raised. The internal-build failure is a CI credentials problem (git fetch of edgeworker master got "HTTP Basic: Access denied" in x64-debug-build), not this change, so a re-run should clear it.

@gabivlj
gabivlj enabled auto-merge November 10, 2026 18:35

@clanky-the-clanker clanky-the-clanker Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Approved — No findings. The patches have not changed since the last review, which found no open issues, and the discussion threads have been addressed.

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


✅ All checks passed

🔒 Security: ✅ LGTM

No security issues found.


📖 AGENTS.md: ✅ LGTM

No AGENTS.md issues found.


📋 Code Quality: ✅ LGTM

pumpTo() and tryPumpFrom() delegate correctly. In each connect state, a failed pump rejects with the stored disconnect exception. The tests cover both the up and down redirect paths.


⚡ Performance: ✅ LGTM

Container port pumps no longer make the extra buffered copy, and the failure handling adds no work to successful pumps.


📝 Documentation: ✅ LGTM

No documentation issues found.


Reviewed commit: 7e866a47

@gabivlj
gabivlj merged commit b78126e into main Nov 10, 2026
23 of 25 checks passed
@gabivlj
gabivlj deleted the gv/container-port-pump-v2 branch November 10, 2026 23:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants