Repository navigation
Count queued outgoing WebSocket messages as isolate memory - #8306
Conversation
There was a problem hiding this comment.
Superseded by the review of
b38dc4c5.
This review, of 470ee8c9
💬 Approved with comments — One WARNING: the comments in pump() and hibernatable-adapter.h say the queue's storage is freed after draining, but the code calls clear(), which keeps it, and nothing charges that storage. Also one INFO about missing tests.
Findings: 1 warning · 2 info (3 are inline comments on the diff)
The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.
🔧 Fix with your agent
Paste the following into your coding agent to investigate and fix these issues.
Fix the following code review issues found in pull request #8306.
Before making changes, review each issue and present a brief summary table:
- For each finding, state whether you agree, disagree, or need clarification
- If you disagree (e.g., the fix would require disproportionate changes
for minimal ROI, or the finding is factually incorrect), explain why
- If you need to clarify something before deciding, ask those questions
- Then share your plan for which issues to tackle and in what order
## WARNING (1)
### [Code Quality] The comment says the outgoing table is replaced rather than cleared, so its storage is freed. The code calls `outgoingMessages.clear()`, which keeps the storage, and nothing charges it after a burst.
- **File:** src/workerd/api/web-socket.c++:1373
## INFO (2)
### [Code Quality] The new memory accounting behind the autogate has no tests.
- **File:** src/workerd/api/web-socket.c++:1481
### [Documentation] This comment says the legacy adapter frees the queue's storage once drained, but the legacy pump only calls `clear()`.
- **File:** src/workerd/api/hibernatable-adapter.h:20
📋 Code Quality: ⚠️ 1 warning · 1 info
The memory accounting is sound, but a comment describes a table replacement the code never does, and the gated behaviour has no tests.
Recommendation: 📝 Minor Comments
- [WARNING] The comment says the outgoing table is replaced rather than cleared, so its storage is freed. The code calls
outgoingMessages.clear(), which keeps the storage, and nothing charges it after a burst. (src/workerd/api/web-socket.c++:1373) - [INFO] The new memory accounting behind the autogate has no tests. (
src/workerd/api/web-socket.c++:1481)
View full review
- Comment describes storage release that doesn't happen (
src/workerd/api/web-socket.c++:1372-1376). The new comment says "Replace the table rather than clearing it, which would keep its storage". The next line is stilloutgoingMessages.clear();.clear()keeps the table's row and index capacity.- After a burst, that storage stays allocated, and nothing charges it once its messages are gone. The comment claims to avoid exactly this gap.
src/workerd/api/hibernatable-adapter.h:19-21repeats the claim for the legacy adapter, and tells the new adapter to match behaviour that doesn't exist.- To fix it, either release the storage for real (for example, reset the
IoOwn<OutgoingMessagesMap>to a fresh table when the queue drains), or drop the claim from both comments.
- No tests for the gated behaviour. Nothing checks that a peer that never reads hits the memory limit when
WEBSOCKET_OUTGOING_QUEUE_MEMORYis on, or that the charge is released after the pump drains or discards messages.
📝 Documentation: 💬 1 info
The new hibernatable-adapter comment says the legacy adapter frees the queue's storage once drained, but it doesn't.
Recommendation: 💬 Optional Nits
- [INFO] This comment says the legacy adapter frees the queue's storage once drained, but the legacy pump only calls
clear(). (src/workerd/api/hibernatable-adapter.h:20)
View full review
src/workerd/api/hibernatable-adapter.h:19-21tells the new adapter to free the queue's storage once drained, "as the legacy adapter does".- The legacy adapter's cleanup calls
outgoingMessages.clear(), which keeps the storage. - Update this comment together with the one in
pump().
Reviewed commit: 470ee8c9
470ee8c to
b38dc4c
Compare
There was a problem hiding this comment.
Superseded by the review of
f9dcb38a.
This review, of b38dc4c5
💬 Approved with comments — The comment that didn't match the code is fixed, since the queue's storage is now actually freed after draining. One INFO note remains: no tests cover the accounting behind the autogate.
Findings: 1 info (1 is an inline comment on the diff)
The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.
📋 Code Quality: 💬 1 info
The comment mismatch in pump() is fixed. The memory accounting behind the autogate still has no tests.
Recommendation: 💬 Optional Nits
- [INFO] No tests cover the new WEBSOCKET_OUTGOING_QUEUE_MEMORY accounting. (
src/workerd/api/web-socket.c++:1481)
View full review
-
Fixed:
pump()now replaces the outgoing table withoutgoingMessages = OutgoingMessagesMap();(src/workerd/api/web-socket.c++:1376), so the comment above it matches the code. Destroying the old rows also releases theirExternalMemoryAdjustmentcharges. -
Still open (INFO): no test checks either of these with
WEBSOCKET_OUTGOING_QUEUE_MEMORYenabled:- a peer that never reads eventually hits the isolate memory limit;
- the charge is released once the pump writes or discards the queued messages.
The
@all-autogatestest variant would run this path.
✅ All other checks passed
📝 Documentation: ✅ LGTM
The note in hibernatable-adapter.h that the legacy adapter frees the queue's storage once drained is now accurate.
Reviewed commit: b38dc4c5
There was a problem hiding this comment.
Superseded by the review of
7cab91e7.
This review, of f9dcb38a
💬 Approved with comments — The only change since the last review is the capnproto bump, and it is consistent across both Bazel files. One INFO note remains: no tests cover the accounting behind the WEBSOCKET_OUTGOING_QUEUE_MEMORY autogate.
Findings: 1 info
The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.
📋 Code Quality: 💬 1 info
The capnproto bump looks fine. One INFO about test coverage is still open.
Recommendation: 💬 Optional Nits
- [INFO] No tests cover the new WEBSOCKET_OUTGOING_QUEUE_MEMORY accounting. (
src/workerd/api/web-socket.c++:1481)
View full review
-
Capnproto bump: the changes to
MODULE.bazel.lockandbuild/deps/gen/deps.MODULE.bazelagree with each other. The commit,strip_prefix, sha256 and lockfileusagesDigestall match. -
Still open (INFO): no test enables
WEBSOCKET_OUTGOING_QUEUE_MEMORYand checks either of these:- a peer that never reads eventually hits the isolate memory limit;
- the charge is released once the pump writes or discards the queued messages.
The
@all-autogatestest variant would run this path, but no test asserts on the result.
✅ All other checks passed
📝 Documentation: ✅ LGTM
The comments added in hibernatable-adapter.h and web-socket.h match the code.
Reviewed commit: f9dcb38a
f9dcb38 to
7cab91e
Compare
There was a problem hiding this comment.
✅ Approved — The web-socket.h changes since the last review are correct: the queued-message charge is released by RAII, the members are declared and destroyed in a safe order, and the comments match the code. No new findings.
The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.
✅ All checks passed
📋 Code Quality: ✅ LGTM
Each queued message keeps its ExternalMemoryAdjustment in GatedMessage::memory. The charge stays until the pump has written or discarded the message, then RAII releases it. This also covers abort, a write that throws, and the queue being reset.
outgoingMemory is declared before outgoingMessages, so the queued adjustments are destroyed before the target they point to. With the gate off, chargeOutgoing() returns kj::none and nothing is charged.
One note from earlier reviews is still open: no test enables WEBSOCKET_OUTGOING_QUEUE_MEMORY and checks the accounting.
📝 Documentation: ✅ LGTM
The new comments in web-socket.h match the code.
Reviewed commit: 7cab91e7
Since #8306, messages a WebSocket has queued but not yet written count toward the isolate's memory limit, but an application has no way to tell how much it has queued. A Worker that streams faster than its peer reads can only guess a send rate, and if it guesses wrong, the isolate exceeds its limit and every connection on it drops (#8477). bufferedAmount is the WHATWG WebSockets standard's answer: the number of payload bytes passed to send() that haven't been written to the connection yet. Text counts as its UTF-8 length; Close frames and framing overhead don't count. As the standard requires, data that send() discards because the connection has failed is counted and never subtracted, so the value keeps climbing if an application keeps sending to a dead connection. The count lives with the outgoing queue in the IoContext-owned native state, which the pump already updates without the isolate lock. It is lowered after each write completes, so it also covers the message the pump is currently writing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Since #8306, messages a WebSocket has queued but not yet written count toward the isolate's memory limit, but an application has no way to tell how much it has queued. A Worker that streams faster than its peer reads can only guess a send rate, and if it guesses wrong, the isolate exceeds its limit and every connection on it drops (#8477). bufferedAmount is the WHATWG WebSockets standard's answer: the number of payload bytes passed to send() that haven't been written to the connection yet. Text counts as its UTF-8 length; Close frames and framing overhead don't count. As the standard requires, data that send() discards because the connection has failed is counted and never subtracted, so the value keeps climbing if an application keeps sending to a dead connection. The count lives with the outgoing queue in the IoContext-owned native state, which the pump already updates without the isolate lock. It is lowered after each write completes, so it also covers the message the pump is currently writing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No description provided.