Repository navigation
Let container port pumps reach the capnp streams - #8273
Conversation
There was a problem hiding this comment.
Superseded by the review of
b33bdaf1.
This review, of e1187014
💬 Approved with comments — The pump delegation and the disconnect-exception bookkeeping look correct. One WARNING: a pump that fails while Port.connect() is outstanding now waits for connect() to settle, and that wait can stall if connect() stays outstanding for the tunnel's lifetime.
Findings: 1 warning (1 is an inline comment on the diff)
The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.
🔧 Fix with your agent
Paste the following into your coding agent to investigate and fix these issues.
Fix the following code review issues found in pull request #8273.
Before making changes, review each issue and present a brief summary table:
- For each finding, state whether you agree, disagree, or need clarification
- If you disagree (e.g., the fix would require disproportionate changes
for minimal ROI, or the finding is factually incorrect), explain why
- If you need to clarify something before deciding, ask those questions
- Then share your plan for which issues to tackle and in what order
## WARNING (1)
### [Code Quality] A pump failure while Port.connect() is outstanding is held until connect() settles. It can stall if connect() stays outstanding for the tunnel's lifetime, which the removed test comment described.
- **File:** src/workerd/api/container.c++:1207
📋 Code Quality: ⚠️ 1 warning
The pump delegation is sound. A pump failure during the connecting window waits on Port.connect(), which could stall depending on when the real service returns.
Recommendation: 📝 Minor Comments
- [WARNING] A pump failure while Port.connect() is outstanding is held until connect() settles. It can stall if connect() stays outstanding for the tunnel's lifetime, which the removed test comment described. (
src/workerd/api/container.c++:1207)
View full review
The new pumpTo() and tryPumpFrom() overrides delegate correctly to down and up. notifyUncleanDisconnect() keeps only the first exception it sees, as intended.
A pump failure while connect() is outstanding waits for connect() (src/workerd/api/container.c++:1202-1210)
propagatePumpFailure()handles every pump failure as an unclean disconnect and returnsrejectWhenDisconnected(). WhileconnectStateisPENDING, that promise rejects only afterPort.connect()settles.- Take a fresh connection where the destination of
pumpTo()breaks, for example a client that went away. That failure used to reach the caller at once. Now it is held until connect() returns. - This pull request rewrites the test comment at
container-test.c++:1350. The old comment said the real container service keepsPort.connect()outstanding for the lifetime of the connection. The new one says it returns once the container accepts the connection. - If the old description is right, two things follow:
- The held pump failure is released only when the tunnel is torn down, and the caller waiting on the pump may be what keeps the tunnel open. That can stall.
tryPumpFrom()never takes thegetSubstream()path on a fresh connection.
- The tests cover two cases: a connect() that fails while a pump is outstanding, and a reused tunnel. None covers a pump failure while connect() is outstanding that is followed by a successful connect().
- Please confirm when the real service returns from
Port.connect(). If it can stay outstanding after the tunnel is usable, one option is to propagate failures that did not come fromdownorupright away.
✅ All other checks passed
🔒 Security: ✅ LGTM
No security issues found.
📖 AGENTS.md: ✅ LGTM
No AGENTS.md issues found.
⚡ Performance: ✅ LGTM
No performance issues found.
📝 Documentation: ✅ LGTM
No documentation issues found.
Reviewed commit: e1187014
danlapid
left a comment
There was a problem hiding this comment.
The state handling looks right to me, and I couldn't find a correctness bug. My comments are mostly about what the tests prove, plus two descriptions that don't match production.
I checked the production side, since the up redirect depends on it. cloudchamber's byteStreamWrite (cc/cloudchamber/go/pkg/capnp/bytestream.go) implements getSubstream(), including limits, ended/reachedLimit and nested substreams. portCapnp.Connect returns only after the container has accepted the connection. So the connectState == SETTLED check only blocks the redirect for the first request on a new tunnel, and requests that reuse a pooled tunnel get it.
One correction to the PR description: it says that after down is redirected, "the container's bytes no longer pass through this process". C++ capnp has no three-party handoff, so the container's writes still reach this process and the RPC layer forwards them to the destination. What's saved is the extra copy and the 4 KiB read/write loop, not the hop through this process. It's worth rewording so nobody relies on the stronger claim.
e118701 to
b33bdaf
Compare
There was a problem hiding this comment.
Superseded by the review of
828e94a5.
This review, of b33bdaf1
✅ Approved — No open findings. The earlier warning about a pump failure waiting for Port.connect() is resolved: the discussion confirms that connect() returns once the container accepts the connection, and the code matches that.
The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.
✅ All checks passed
🔒 Security: ✅ LGTM
No security issues found.
📖 AGENTS.md: ✅ LGTM
No AGENTS.md issues found.
📋 Code Quality: ✅ LGTM
The pump overrides delegate correctly, and the disconnect-exception bookkeeping holds up. The earlier concern about a pump failure being held until Port.connect() settles no longer applies: the service returns connect() once the container accepts the connection, as confirmed in the discussion, and the existing disconnect signals already wait in the same way. The new tests cover both the up and down redirect paths and check that the body arrives.
⚡ Performance: ✅ LGTM
The change removes the extra buffered copy on container port pumps. The failure-handling path adds nothing to successful pumps.
📝 Documentation: ✅ LGTM
No documentation issues found.
Reviewed commit: b33bdaf1
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #8273 +/- ##
==========================================
+ Coverage 65.46% 65.48% +0.02%
==========================================
Files 759 759
Lines 192409 192659 +250
Branches 30344 30375 +31
==========================================
+ Hits 125951 126158 +207
- Misses 48841 48861 +20
- Partials 17617 17640 +23 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Pumps through a container port, such as a fetch() request body or a connect() socket, were slower than they need to be, because ContainerPortAsyncIoStream did not override pumpTo() or tryPumpFrom(). A pump from the port read `down` in a generic loop, 4 KiB at a time with an extra copy, or 8 KiB into each RPC message when the destination was a capnp ByteStream. A pump into the port used the source's fallback, often the same 4 KiB loop. Neither direction could redirect a capnp ByteStream to the other side with getSubstream(). pumpTo() now delegates to the `down` pipe. Each write the container makes on `down` goes straight to the destination, and when the destination is a capnp ByteStream, `down` can be redirected to it with getSubstream(). tryPumpFrom() now delegates to the `up` ByteStream once Port.connect() has returned. The adapter reads the source straight into RPC messages or, when the source is itself a capnp ByteStream, redirects it to `up` with getSubstream(). C++ capnp has no three-party handoff, so redirected bytes still reach this process, and the RPC layer forwards them to the other stream. A redirect saves the copy and the read/write loop, not the hop through this process. fetch() and connect() still fail with Port.connect()'s error when the tunnel fails to open: - Until Port.connect() has returned, tryPumpFrom() returns none and the caller falls back to write(), as before. If the tunnel fails to open, a redirected source's getSubstream() fails and the source's pump is cancelled with a generic error that does not pass through this stream, so it could arrive ahead of Port.connect()'s error. Pumps that start before Port.connect() returns, such as the first request body on a new tunnel, are therefore written as before; a request body on a reused tunnel can be redirected. - A failed pump is handled as an unclean disconnect and rejects with the stream's disconnect exception. While Port.connect() is outstanding this waits for its result, like the existing disconnect signals, so a pump broken by the tunnel failing to open reports Port.connect()'s error. notifyUncleanDisconnect() now keeps the exception of the first such signal, which is used if Port.connect() succeeds. Other behaviour changes: - Every failed pump now tears down the tunnel, including failures caused by the other stream, such as a request body that errors or a destination that rejects writes. The connection cannot be reused after a partial pump anyway. - If the sender of a body that is redirected to `up` aborts, the caller sees capnp's "stream disconnected because SubstreamCallbackImpl was never called back" instead of the body stream's own error, and that becomes the tunnel's disconnect exception. The pooled HTTP path can hit this when a client aborts an upload. - Cancelling a redirected pump does not revoke the substream, so the sender can keep writing into `up` after this side has given up on the connection. This is inherent to redirecting with getSubstream(); the write-based pump did not do it. - When a raw connect()'s `down` is redirected, for example because the Worker pumps the socket's readable into a stream sent over RPC, an unclean end of the tunnel fails that pump with a generic error. Before, the readable ended, because the HTTP client adapter shuts the socket's pipe down when connect() fails after the socket was accepted. The container mock now reads request bodies, so tests can check what reached `up`. The new tests pump a request body while Port.connect() is outstanding and then fails, with getSubstream() failing with a different error; pump one on a reused tunnel, where it is redirected to `up` and arrives intact; pump a raw connect()'s socket into a capnp ByteStream, to which `down` is redirected; and drop `down` after that redirect before Port.connect() fails.
b33bdaf to
828e94a
Compare
There was a problem hiding this comment.
Superseded by the review of
7e866a47.
This review, of 828e94a5
✅ Approved — No open findings. The test changes since the last review fix the coverage gaps raised in the discussion, and the production pump logic still holds up.
The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.
✅ All checks passed
🔒 Security: ✅ LGTM
No security issues found.
📖 AGENTS.md: ✅ LGTM
No AGENTS.md issues found.
📋 Code Quality: ✅ LGTM
pumpTo() and tryPumpFrom() delegate correctly. In each connect state, a failed pump rejects with the stored disconnect exception. The updated tests use a distinct getSubstream() error, check that the body reached up, implement tryReadSync() on the mock input, and cover the down redirect.
⚡ Performance: ✅ LGTM
Container port pumps no longer make the extra buffered copy, and the failure handling adds no work to successful pumps.
📝 Documentation: ✅ LGTM
No documentation issues found.
Reviewed commit: 828e94a5
danlapid
left a comment
There was a problem hiding this comment.
Thanks, this addresses everything I raised. The internal-build failure is a CI credentials problem (git fetch of edgeworker master got "HTTP Basic: Access denied" in x64-debug-build), not this change, so a re-run should clear it.
There was a problem hiding this comment.
✅ Approved — No findings. The patches have not changed since the last review, which found no open issues, and the discussion threads have been addressed.
The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.
✅ All checks passed
🔒 Security: ✅ LGTM
No security issues found.
📖 AGENTS.md: ✅ LGTM
No AGENTS.md issues found.
📋 Code Quality: ✅ LGTM
pumpTo() and tryPumpFrom() delegate correctly. In each connect state, a failed pump rejects with the stored disconnect exception. The tests cover both the up and down redirect paths.
⚡ Performance: ✅ LGTM
Container port pumps no longer make the extra buffered copy, and the failure handling adds no work to successful pumps.
📝 Documentation: ✅ LGTM
No documentation issues found.
Reviewed commit: 7e866a47
Pumps through a container port, such as a fetch() request body or a connect() socket, were slower
than they need to be, because ContainerPortAsyncIoStream did not override pumpTo() or
tryPumpFrom(). A pump from the port read
downin a generic loop, 4 KiB at a time with an extracopy, or 8 KiB into each RPC message when the destination was a capnp ByteStream. A pump into the
port used the source's fallback, often the same 4 KiB loop. Neither direction could redirect a
capnp ByteStream to the other side with getSubstream().
pumpTo() now delegates to the
downpipe. Each write the container makes ondowngoes straightto the destination, and when the destination is a capnp ByteStream,
downcan be redirected to itwith getSubstream().
tryPumpFrom() now delegates to the
upByteStream once Port.connect() has returned. The adapterreads the source straight into RPC messages or, when the source is itself a capnp ByteStream,
redirects it to
upwith getSubstream().fetch() and connect() still fail with Port.connect()'s error when the tunnel fails to open:
write(), as before. If the tunnel fails to open, a redirected source's getSubstream() fails and
the source's pump is cancelled with a generic error that does not pass through this stream, so
it could arrive ahead of Port.connect()'s error. Pumps that start before Port.connect() returns,
such as the first request body on a new tunnel, are therefore written as before; a request body
on a reused tunnel can be redirected.
exception. While Port.connect() is outstanding this waits for its result, like the existing
disconnect signals, so a pump broken by the tunnel failing to open reports Port.connect()'s
error. notifyUncleanDisconnect() now keeps the exception of the first such signal, which is used
if Port.connect() succeeds.
Other behaviour changes:
as a request body that errors or a destination that rejects writes. The connection cannot be
reused after a partial pump anyway.
upaborts, the caller sees capnp's "streamdisconnected because SubstreamCallbackImpl was never called back" instead of the body stream's
own error, and that becomes the tunnel's disconnect exception. The pooled HTTP path can hit this
when a client aborts an upload.
upafter this side has given up on the connection. This is inherent to redirecting withgetSubstream(); the write-based pump did not do it.
downis redirected, for example because the Worker pumps the socket'sreadable into a stream sent over RPC, an unclean end of the tunnel fails that pump with a
generic error. Before, the readable ended, because the HTTP client adapter shuts the socket's
pipe down when connect() fails after the socket was accepted.
The container mock now reads request bodies, so tests can check what reached
up. The new testspump a request body while Port.connect() is outstanding and then fails, with getSubstream()
failing with a different error; pump one on a reused tunnel, where it is redirected to
upandarrives intact; pump a raw connect()'s socket into a capnp ByteStream, to which
downisredirected; and drop
downafter that redirect before Port.connect() fails.