Skip to content

Port MemoryCache to Rust, behind an autogate - #8066

Merged
bartlomieju merged 27 commits into
mainfrom
biwanczuk/rust-memory-cache-read
Nov 12, 2026
Merged

bartlomieju merged 27 commits into
mainfrom
biwanczuk/rust-memory-cache-read

Conversation

@bartlomieju

@bartlomieju bartlomieju commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Port MemoryCache (read() and delete()) to Rust JSG, behind a new
RUST_MEMORY_CACHE autogate. With the autogate on, workerd-api.c++
creates the Rust MemoryCache for memory cache bindings, and
MemoryCacheProvider::bindForRust() gives it the same shared namespaces
that the C++ one uses.

src/workerd/api/memory-cache-api sits on top of the cache in
src/rust/memory-cache, and uses the io-context crate for I/O, objects
owned by the request and trace spans:

  • lib.rs: read(), delete(), key validation, and cache hits.
  • fallback.rs: a read that waits as the leader (until it runs its
    fallback) or as a waiter (until the read it waits for finishes), and
    the promise chain that runs the fallback and caches what it produces.
  • trace.rs: the spans and tags that memory-cache.c++ records.

It is meant to be indistinguishable from the C++ one, under every
compatibility date and autogate:

  • Arguments and errors: errors and messages are the same. read()
    throws, or rejects under capture_async_api_throws, and always rejects
    for a key that is too large. delete() requires a request, records the
    same span, and exists only under memory_cache_delete.
  • The fallback is called with globalThis as this, and with the key
    decoded from the UTF-8 bytes the cache holds, so a lone surrogate
    reaches it as U+FFFD.
  • What the fallback produces is converted as C++ JSG converts a
    jsg::Promise. A promise is followed, and a
    thenable only under unwrap_custom_thenables. Anything else is read as
    the fallback returns, and an error reading it rejects the read as a
    throw would. Values that are not objects fail with C++'s messages,
    except that the type is named CacheValueProduceResult, as
    jsg::typeName() names it, rather than mangled.
  • If serializing the value throws, the read rejects with what C++ gets
    by tunneling the exception through a kj::Exception: a copy of the
    exception, built from its serialized form only under
    enhanced_error_serialization, or, for an exception with an empty
    detail string such as the serializer's own error, a DataCloneError
    that names the type.
  • Every read settles after as many microtasks as in C++.
  • A fallback whose request ends is abandoned with that request, so
    another read runs its own fallback rather than waiting for the garbage
    collector to drop the promise. A fallback whose promise another
    request settles behaves as in C++, under
    handle_cross_request_promise_resolution and without it. A read
    resumes inside a Durable Object's blockConcurrencyWhile() critical
    section.
  • Spans and tags are the same as in C++, and cache errors become
    internal errors.
  • Releasing a binding reads the request's clock if a request is
    current, and the system clock otherwise, as cacheNow() does, to tell
    which entries have expired.

One difference remains. Without handle_cross_request_promise_resolution,
a fallback whose promise another request settles rejects the read with
"Cannot perform I/O on behalf of a different request." in both
implementations. C++ appends " (I/O type: SpanBuilder&)>)", which is
jsg::typeName() of the callback its IoOwn holds; the Rust message has no
suffix. The test checks the whole message, allowing only that suffix.

The io-context crate gains IoContext::now_in_current_request(), which
reads the current request's clock without a lock, as C++ dateNow()
does, for code that holds none, such as a Drop impl.

Rust JSG additions:

  • Local::to_detail_string() describes a value as C++ kj::str() of a
    v8::Local does, without running JavaScript.
  • Lock::global_object() returns globalThis.
  • jsg::TunneledException converts a JavaScript exception to a
    kj::Exception and back, as Lock::exceptionToKj() and exceptionToJs()
    do. It reads a DOMException through
    jsg::tryGetDOMExceptionNameAndMessage(), a hook beside
    newDOMException() that takes and returns only V8 and KJ types, built
    on a new jsg::Lock::tryUnwrapDOMException().
  • jsg::ser::deserialize_with() can drop the stacks of deserialized
    errors. jsg::ser::deserialize() keeps them, as before.
  • Lock::throw_value() throws any value, and
    Lock::new_object_with_null_prototype() creates an object that
    resolving a promise with follows nothing.

Testing:

  • memory-cache-rust-test runs the services of memory-cache-test with
    the autogate on, so every variant covers the Rust MemoryCache, as
    @all-autogates covers it for memory-cache-test. That includes the
    newest compatibility date and @gc-stress, which, like every
    @gc-stress variant, runs only when requested (off-by-default).
  • New JS tests cover what fallbacks produce and thenables, getters that
    throw, microtask counts, a fallback settled by another request, the
    fallback's receiver, key and async context, abandoned fallbacks,
    blockConcurrencyWhile(), throwing versus rejecting, and delete().
  • A test in memory-cache-test.c++ runs one scenario against both
    implementations and checks that they record the same spans.
    SpanRecorder now records spans in the order they open.
  • jsg::TunneledException is checked against C++ JSG's round trip, with
    and without enhanced_error_serialization.
  • io-context-test checks now_in_current_request() against a request
    clock that TestFixture's new fixedNow option fixes, so it differs
    from the system clock.

The autogate is off by default, and with it off memory cache bindings
are the C++ MemoryCache, created as before, so no Worker sees a change
until it is enabled.

@codecov-commenter

codecov-commenter commented Oct 28, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.36533% with 129 lines in your changes missing coverage. Please review.
✅ Project coverage is 66.28%. Comparing base (22a428d) to head (e1cb715).

Files with missing lines Patch % Lines
src/rust/jsg/ffi.c++ 40.74% 18 Missing and 14 partials ⚠️
src/workerd/api/memory-cache-api/fallback.rs 90.03% 27 Missing ⚠️
src/rust/jsg/tunnel.rs 86.55% 16 Missing ⚠️
src/workerd/api/memory-cache-test.c++ 83.83% 3 Missing and 13 partials ⚠️
src/workerd/io/io-context/io-context-test.c++ 74.60% 0 Missing and 16 partials ⚠️
src/workerd/api/memory-cache-api/lib.rs 94.65% 7 Missing ⚠️
src/workerd/jsg/dom-exception.c++ 40.00% 5 Missing and 1 partial ⚠️
src/workerd/io/io-context/lib.rs 88.88% 4 Missing ⚠️
src/workerd/api/memory-cache-api/trace.rs 96.62% 3 Missing ⚠️
src/rust/jsg/ser.c++ 75.00% 0 Missing and 1 partial ⚠️
... and 1 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8066      +/-   ##
==========================================
+ Coverage   66.01%   66.28%   +0.26%     
==========================================
  Files         776      781       +5     
  Lines      197923   198892     +969     
  Branches    31057    31114      +57     
==========================================
+ Hits       130663   131834    +1171     
+ Misses      49193    48914     -279     
- Partials    18067    18144      +77     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@guybedford guybedford left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review suggestions:

  • Argument errors reject instead of throwing (lib.rs:62). C++ JSG throws synchronously when an argument has the wrong type, so CACHE.read(123) and CACHE.read('k', 5) throw in C++ but return a rejected promise in Rust. Only "Key too large." and later failures should reject.
  • delete() records no memory_cache_delete span. C++ records one with key and delete_completed. The parity scenario never calls delete, so the span parity test doesn't catch it.
  • A thenable or non-object fallback result behaves differently (fallback.rs:177, fallback.rs:315). If the fallback returns { then(r) { r({ value: 'th' }) } }, C++ resolves undefined (it treats the object itself as the result) and Rust resolves 'th'. If it returns 42, C++ rejects with "Callback returned incorrect type; expected 'Promise'" and Rust with "The fallback must produce an object with a value property."
  • Errors thrown while serializing keep their original identity in Rust (fallback.rs:292). If a getter on the value throws a MyErr, C++ rejects with a new plain Error and Rust rejects with the original MyErr. A thrown string becomes an Error in C++ but stays a string in Rust, and the fallback_error tag changes with it. Rust's behaviour is arguably better, but it's a divergence to either accept explicitly or match.
  • A JS-level parity test with these cases, run in both the @ and @all-autogates variants, would back up the parity claim for what JS sees and not only for the spans.

bartlomieju added a commit that referenced this pull request Oct 30, 2026
…8030)

Give Rust APIs the two IoContext operations that porting
MemoryCache.read() to Rust (#8066) needs: await_io_and_run(), to wait
for I/O and then resume the request, and add_object(), to tie a value's
lifetime to the request.

await_io_and_run() wraps a new C++ IoContext::awaitIoAndRun(), which
shares its implementation with awaitIo() but returns no promise: the
callback settles whatever it needs to itself, and failures fail the
task as for addTask(). add_object() wraps addObject() and returns an
IoOwned with IoOwn's lifetime rules. These replace the
unimplemented jsg::Lock::await_io() stub, and jsg::Error's
new_internal_error() becomes public for the port to use.

Tests cover both operations in requests to a real worker, including
Durable Objects, failures and panics. No JS-visible change; nothing
outside tests uses these yet.
Base automatically changed from biwanczuk/rust-io-context-trace-spans to main November 2, 2026 20:12
Native code that holds no jsg::Lock, such as Rust JSG's shims, can create
a DOMException with any name through it. It takes and returns only V8 and
KJ types, so its callers don't depend on how DOMException is implemented.

newDataCloneError() becomes a wrapper around it, unchanged in behaviour.
Rust JSG's DOMException error types (AbortError, DataCloneError,
NotSupportedError, ...) were thrown as a plain Error. They are now thrown
as a DOMException with that name, as C++ JSG_REQUIRE(..., DOMAbortError,
...) throws, created through jsg::newDOMException(). Lock::dom_exception()
creates one with any name, as jsg::Lock::domException() does.

Creating a DOMException needs the current context, unlike creating an
Error. So while an exception or a termination is pending, or if creating
it throws, Lock::exception() and Lock::dom_exception() fail with a
pending error and Lock::throw_exception() throws nothing, so the pending
exception propagates. Error::to_local() now returns a Result for this.
A fast callback never throws one: only methods returning no Result have
a fast callback, and they throw only TypeErrors (now debug-asserted).

SyntaxError stays the JavaScript SyntaxError. A DOMException named
SyntaxError needs its own ExceptionType variant, which is left for later.

Compatibility: no shipped Rust API returns a DOMException error type, so
no Worker sees a change.

Safety rule 3 lists newDOMException() beside the ser.h hooks that Rust
shims may call.
bartlomieju added a commit that referenced this pull request Nov 4, 2026
A #[jsg_struct] field can now hold a JavaScript value, as a C++
JSG_STRUCT field of type jsg::JsRef<T> does. Such a field is a
jsg::v8::TracedGlobal<M>, which now implements Type, ToJS and FromJS
and converts as a v8::Local<'s, M> does:

- TracedGlobal<Value> takes any value, and a missing property reads as
  undefined, as with jsg::JsRef<jsg::JsValue>.
- TracedGlobal<Object> (and the other markers with a FromJS for their
  Local) takes only a value of that type, with the Local's TypeError
  ("Expected Object but got number").
- Converting to JavaScript gives back the value itself.

The macro needed no change: it already converts each field through its
own type and traces every field. So a struct with such a field holds GC
references, and a resource that keeps it traces the value, so a cycle
through it is collected.

Porting MemoryCache (#8066) needs this. Its fallback produces a
CacheValueProduceResult, whose value is a jsg::JsRef<jsg::JsValue>.
The Rust port reads that object by hand today. With this change, and
with typed promises (#8153), it can be a struct carried by a
jsg::Promise<'s, T>. That needs a 'static T, so the field is a
TracedGlobal and not a Local with the struct's lifetime.

Tests cover keeping the value's identity (with and without
fast_jsg_struct), any value and missing properties, a rejected
non-object, a struct kept past the scope it was converted in, and the
collection of a cycle through such a field. No JS-visible change;
nothing outside tests uses this yet.
Wraps v8::Value::ToDetailString(), which describes a value as C++
kj::str() of a v8::Local does, without running JavaScript such as a
toString() method. The Rust MemoryCache uses it for the error tag
of a failed fallback's trace span, which must match the C++ one.
Adds src/workerd/api/memory-cache-api, a Rust MemoryCache over the
cache in src/rust/memory-cache, built on the io-context crate's
await_io_and_run(), add_object() and trace spans. With the
RUST_MEMORY_CACHE autogate enabled, memory cache bindings are the
Rust MemoryCache; otherwise they stay the C++ one.

MemoryCacheProvider::bindForRust() hands the Rust MemoryCache the
same shared namespaces that the C++ one uses.

A new test in memory-cache-test.c++ runs reads that cover every
path of read() against both implementations, and checks they
record the same spans. SpanRecorder now records spans in the order
they open, since the order they end in depends on when each
implementation releases them.

New JS tests cover a fallback abandoned when its request ends, the
async context a fallback runs in, the key a fallback gets for a
lone surrogate, a NaN expiration, and a fallback inside a Durable
Object's blockConcurrencyWhile(). They run against both
implementations through the @all-autogates variant.
Returns the global object of the entered context, as C++ JSG's
jsg::Lock::global() does. C++ JSG calls a jsg::Function with it as
the receiver; the Rust MemoryCache needs it to call a fallback the
same way.
- delete() records a memory_cache_delete span tagged with the key
  and delete_completed, and so, like C++, fails outside a request.
- The fallback is called with globalThis as `this`, as C++ JSG calls
  a jsg::Function, rather than undefined.
- A key that is not a string names the method that got it, and a
  fallback that is not a function is "not of type 'function'", as in
  C++.

The parity scenario in memory-cache-test.c++ now deletes a cached key
and one never cached, and runs with memory_cache_delete on. New JS
tests cover the fallback's receiver, the TypeErrors for invalid
arguments, and delete() in the global scope; each fails against the
Rust implementation without these changes.
JSG calls a C++ method that returns a promise through
LiftKj_<v8::Local<v8::Promise>>: under capture_async_api_throws (on
by default since 2022-10-31) what the method throws rejects the
promise, and otherwise it is thrown. The Rust read() rejected in both
cases. It now throws unless the flag is on, and under the flag catches
what it throws, a JavaScript exception included, and rejects with it,
as LiftKj_ does. A key that is too large still rejects either way,
since memory-cache.c++ rejects it explicitly.

memory-cache-throws-test.js checks, for a key that is not a string, a
fallback that is not a function and a read in the global scope, that
read() throws or rejects as the flag says. It also checks that a key
that is too large rejects and that delete() throws whatever the flag.
memory-cache-test.wd-test runs it in two services, one with the flag
as the compatibility date sets it and one that enables it, so the
@ and @all-autogates variants cover both settings against both
implementations. Without this change, the Rust implementation fails it
with the flag off.

Also restores the TODO(#7951) in memory-cache-test.c++ to its
original form.
…+ does

With Rust JSG throwing DOMExceptions, the Rust MemoryCache rejects a
fallback's value that cannot be serialized with a DataCloneError
DOMException, as the C++ one does. The parity scenario in
memory-cache-test.c++ now includes such a value, so both
implementations record the same memory_cache_fallback tags for it,
and the JS test checks the rejection's name, code and that it is a
DOMException.
@bartlomieju
bartlomieju force-pushed the biwanczuk/rust-memory-cache-read branch from f73a613 to 6ace636 Compare November 4, 2026 11:50
@bartlomieju

Copy link
Copy Markdown
Collaborator Author

Rebased onto #8210 (Throw DOMExceptions from Rust JSG), so this branch now carries its two commits until it lands. With it, the Rust MemoryCache rejects an unserializable value with a DataCloneError DOMException, and the parity test covers that case.

Differences to fix before merging

I compared C++ and Rust on about 40 observable behaviours of env.CACHE, at compat dates 2000-01-01 and 2999-12-31, with and without memory_cache_delete. These still differ:

Case C++ Rust Dates
A The fallback produces something other than an object sync 42: Callback returned incorrect type; expected 'Promise'
async 42: Incorrect type for Promise: the Promise did not resolve to 'CacheValueProduceResult'.
undefined/null: Cannot initialize N7workerd3api23CacheValueProduceResultE with required members from an undefined or null value.
always The fallback must produce an object with a value property. all
B The fallback returns a thenable that also has a value, e.g. { then(r) { r({ value: 'from then' }) }, value: 'own' } resolves 'own' without unwrap_custom_thenables (before 2024-04-01), 'from then' with it always 'from then' before 2024-04-01
C Serializing the value throws from a getter (throw new Custom()) rejects with a copy: a plain Error, not instanceof Custom rejects with the original object all
D Settle timing of a hit, a miss and Key too large after 2 microtasks after 1 all

The leader and waiter paths cross an event-loop turn, so their microtask counts still need measuring.

Coverage to add before merging

  • Rust at the newest compat date. @all-autogates runs only at 2000-01-01, so apart from the capture_async_api_throws service, no test runs Rust with flags enabled by date.
  • Rust under GC stress. memory-cache-test@gc-stress has no autogates, so it runs only C++.
  • memory_cache_delete off. Nothing checks that delete is absent.
  • A fallback whose promise settles in another request (handle_cross_request_promise_resolution). Not tested against either implementation.

The plan for the first three is a second wd_test whose config enables workerd-autogate-rust-memory-cache, so @, @all-compat-flags and @gc-stress all run the JS tests against Rust. It gets deleted with the autogate.

Open question

Whatever Rust copies from C++ for A and C becomes its permanent behaviour once the C++ implementation is removed, including the mangled type name in A's message. Should we copy these exactly, or accept them as differences?

…riant

Add memory-cache-rust-test, which runs the services of memory-cache-test
with the rust-memory-cache autogate on, so the newest compatibility date
and GC stress cover the Rust MemoryCache, not only @all-autogates.

Check that delete() is exposed only under memory_cache_delete.
C++ JSG converts the value a fallback returns to a
jsg::Promise<CacheValueProduceResult>: it takes a promise as is, follows
a thenable only under unwrap_custom_thenables, and reads anything else as
the fallback returns, so that what reading it throws rejects the read as
if the fallback threw it. The Rust MemoryCache now does the same, and
fails with the same messages for a value that is not an object, except
that it names CacheValueProduceResult as jsg::typeName() does rather than
mangled.
Add jsg::ser::deserialize_with() and DeserializeOptions, with the
preserveStackInErrors option of jsg::Deserializer, which C++ JSG turns
off when it reads an exception tunnelled through a kj::Exception.
C++ code that catches a JavaScript exception and throws it on as a
kj::Exception hands JavaScript a copy: Lock::exceptionToKj() describes
the exception and serializes it, and exceptionToJs() rebuilds it from
that. TunneledException does the same in Rust, so that Rust ports of
such C++ code reject with the same copy. A test compares its copies
with C++ JSG's for errors of each type, DOMExceptions, non-errors and
the overloaded and retryable hints, with and without
enhanced_error_serialization.

Rust JSG reads a DOMException's name and message through
jsg::tryGetDOMExceptionNameAndMessage(), a hook beside newDOMException()
that unwraps it as C++ JSG does, including from a prototype chain.
memory-cache.c++ throws on what serializing the value throws as a
kj::Exception, so the read rejects with the copy that tunnelling makes,
and with a DataCloneError naming the type for an exception whose detail
string is empty. The Rust MemoryCache now does the same, through
jsg::TunneledException, rather than rejecting with the exception
itself.
throw_value() throws any JavaScript value, as a reaction to a promise
needs to when it rejects with an exception it caught, rather than
returning a rejected promise, which takes two more microtasks to follow.

new_object_with_null_prototype() creates an object that a promise can be
resolved with without looking up a then, as C++ JSG resolves a promise
with a C++ value.
Follow the promise chain of memory-cache.c++ step for step: JSG's
reaction to the promise read() returns, the reaction that reads the
fields of a promised fallback result, and separate .then() and .catch_()
reactions that throw rather than return rejected promises, with the
fallback's result passed on in a null-prototype object as JSG passes a
C++ value. A read from the cache resolves the read with a promise, as
awaitIo() resolves it with the one memory-cache.c++ returns.

A test checks the microtasks a read takes to settle, on a hit, a miss, a
key that is too large and each way a fallback can produce or fail.
Under handle_cross_request_promise_resolution, the reactions to it run in
the request that read, which caches the value. Before it, they run in the
request that settled it, where the fallback is out of reach, so the read
fails and caches nothing. The Rust MemoryCache does the same.
@bartlomieju bartlomieju changed the title Port MemoryCache.read() to Rust, behind an autogate Port MemoryCache to Rust, behind an autogate Nov 4, 2026
#8210 landed squashed, with follow-ups to the two commits of it that this
branch carries, so the conflicts in the DOMException and Rust JSG files
take its final form and keep this branch's additions on top
(tryGetDOMExceptionNameAndMessage() and its Rust shim, and
Lock::throw_value()). autogate.h keeps both RUST_MEMORY_CACHE and main's
DURABLE_OBJECT_WEBSOCKET_LIMIT_KILLSWITCH.

Adapted to Rust JSG API changes on main:

- jsg::Rc::new() takes the lock (#8111).
- Traced::trace is required, so the types that hold no GC references use
  jsg_traced!, which also marks them HoldsNoGcReferences (#8196).
@bartlomieju
bartlomieju marked this pull request as ready for review November 7, 2026 15:26
@bartlomieju
bartlomieju requested review from a team as code owners November 7, 2026 15:26

@clanky-the-clanker clanky-the-clanker Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by the review of 52d79219.

This review, of 8615a396

💬 Approved with comments — No blocking or warning-level issues. Two INFO notes on small differences from the C++ implementation, behind an autogate that is off by default.

Findings: 2 info (2 are inline comments on the diff)

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


📋 Code Quality: 💬 2 info

No correctness bugs; two minor differences from the C++ implementation.

Recommendation: 💬 Optional Nits

  • [INFO] CacheBinding::drop always uses the system clock; the C++ MemoryCache uses the request clock when a request is current. (src/workerd/api/memory-cache-api/lib.rs:235)
  • [INFO] The cross-request error message lacks the C++ (I/O type: ...) suffix; the test regex checks only the prefix, so it cannot catch the difference. (src/workerd/api/tests/memory-cache-test.js:453)
View full review

The autogate wiring, the namespace sharing through bindForRust(), the new Rust JSG shims and the tests all look correct. Two small differences from the C++ implementation remain:

  • CacheBinding::drop releases the binding using the system clock rather than the request clock.
  • The cross-request I/O error message has no (I/O type: ...) suffix. The pull request description discloses this.

✅ All other checks passed

🔒 Security: ✅ LGTM

  • Namespaces: bindForRust reuses the same namespace binding as the C++ path, so tenant isolation is unchanged.
  • Keys: both read and delete type-check keys and cap them at 2 KiB before the cache is touched.
  • Stacks: dropping error stacks during deserialization only reduces what is exposed, and stacks are still kept by default.
  • Rollout: the feature is behind an autogate that is off by default.

📖 AGENTS.md: ✅ LGTM

No AGENTS.md changes are required.


⚡ Performance: ✅ LGTM

No performance concerns.


📝 Documentation: ✅ LGTM

The README update to the JSG shim safety rules accurately lists the new tryGetDOMExceptionNameAndMessage hook.


Reviewed commit: 8615a396

Comment thread src/workerd/api/memory-cache-api/lib.rs Resolved

@clanky-the-clanker clanky-the-clanker Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by the review of b3bd116e.

This review, of 52d79219

💬 Approved with comments — No blocking or warning-level issues. Two INFO notes on small differences from the C++ implementation, behind an autogate that is off by default.

Findings: 2 info

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


📋 Code Quality: 💬 2 info

No correctness bugs; two minor differences from the C++ implementation.

Recommendation: 💬 Optional Nits

  • [INFO] CacheBinding::drop always uses the system clock; the C++ MemoryCache uses the request clock when a request is current. (src/workerd/api/memory-cache-api/lib.rs:235)
  • [INFO] The cross-request error message lacks the C++ (I/O type: ...) suffix; the test regex checks only the prefix, so it cannot catch the difference. (src/workerd/api/tests/memory-cache-test.js:453)
View full review

The autogate wiring, the namespace sharing through bindForRust(), the new Rust JSG shims and the tests all look correct. Two small differences from the C++ implementation remain:

  • CacheBinding::drop releases the binding using the system clock rather than the request clock.
  • The cross-request I/O error message has no (I/O type: ...) suffix. The pull request description discloses this.

✅ All other checks passed

🔒 Security: ✅ LGTM

  • Namespaces: bindForRust reuses the same namespace binding as the C++ path, so tenant isolation is unchanged.
  • Keys: both read and delete type-check keys and cap them at 2 KiB before the cache is touched.
  • Stacks: dropping error stacks during deserialization only reduces what is exposed, and stacks are still kept by default.
  • Rollout: the feature is behind an autogate that is off by default.

📖 AGENTS.md: ✅ LGTM

No AGENTS.md changes are required.


⚡ Performance: ✅ LGTM

No performance concerns.


📝 Documentation: ✅ LGTM

The README update to the JSG shim safety rules accurately lists the new tryGetDOMExceptionNameAndMessage hook.


Reviewed commit: 52d79219 — same changes as 8615a396, so this is that commit's review (no new review was run; line numbers are as of 8615a396)

memory-cache.c++'s cacheNow() reads the request's clock when a request
is current, and the system clock otherwise. CacheBinding's Drop always
read the system clock, since it has no lock to find the request with.

- Add IoContext::now_in_current_request(), which reads the current
  request's clock without a lock, as C++ dateNow() does
- Use it when releasing the binding, falling back to the system clock
- Let TestFixture fix the mock request clock, so a test can tell it
  from the system clock
- Pin the whole cross-request error message in memory-cache-test.js,
  allowing only C++'s "(I/O type: ...SpanBuilder...)" suffix

@clanky-the-clanker clanky-the-clanker Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by the review of 2393ff7c.

This review, of b3bd116e

💬 Approved with comments — The two earlier INFO notes are resolved and nothing new blocks merging. The only new note is an INFO about wording in the Rust JSG README, and the Rust MemoryCache stays behind an autogate that is off by default.

Findings: 1 info (1 is an inline comment on the diff)

Not opened by the correctness review: 1 changed file it was meant to read
  • src/workerd/api/memory-cache-api/lib.rs

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


📝 Documentation: 💬 1 info

One unclear phrase in the Rust JSG README.

Recommendation: 💬 Optional Nits

  • [INFO] Unclear phrase "whatever implements them" in the Rust JSG README. (src/rust/jsg/README.md:253)
View full review

src/rust/jsg/README.md:253: the phrase "DOMExceptions whatever implements them" is hard to parse.


✅ All other checks passed

🔒 Security: ✅ LGTM

  • Namespaces: bindForRust reuses the namespace binding that the C++ path uses, so tenant isolation is unchanged.
  • DOMException hooks: the new hooks run no JavaScript.
  • Stacks: dropping error stacks during deserialization only removes information. Stacks are still kept by default.
  • Rollout: the feature is behind an autogate that is off by default.

📖 AGENTS.md: ✅ LGTM

No AGENTS.md changes are needed.


📋 Code Quality: ✅ LGTM

Both earlier notes are resolved:

  • Release clock: CacheBinding::drop (src/workerd/api/memory-cache-api/lib.rs:233) now uses IoContext::now_in_current_request().unwrap_or_else(SystemTime::now), which matches cacheNow(). io-context-test.c++ pins this with a fixed request clock, so the test would fail if the system clock were used.
  • Cross-request error message: memory-cache-test.js now checks the full message and allows only the (I/O type: …SpanBuilder…) suffix that C++ adds. The pull request description discloses this difference.

What else I checked:

  • Early returns: they release the permit, so no read is left waiting.
  • Termination: nothing is thrown during termination.
  • Autogate: with the autogate off, the wiring in workerd-api.c++ is unchanged.
  • fixedNow: the option defaults to none, so existing tests are unaffected.
  • Tests: each Rust JSG addition has its own test.

⚡ Performance: ✅ LGTM

No performance concerns.


Reviewed commit: b3bd116e

Comment thread src/rust/jsg/README.md Resolved
Co-authored-by: clanky-the-clanker[bot] <342202840+clanky-the-clanker[bot]@users.noreply.github.com>

@clanky-the-clanker clanky-the-clanker Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by the review of 0db1504b.

This review, of 2393ff7c

✅ Approved — No open findings: the only earlier note, about README wording, is fixed. The Rust MemoryCache stays behind an autogate that is off by default.

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


✅ All checks passed

🔒 Security: ✅ LGTM

  • Namespaces: bindForRust() uses the same namespace provider as the C++ path, so cache isolation between bindings is unchanged.
  • DOMException hooks: they run no JavaScript and copy out only the name and message.
  • Rollout: the change is behind an autogate that is off by default.

📖 AGENTS.md: ✅ LGTM

No AGENTS.md changes are needed.


📋 Code Quality: ✅ LGTM

  • Validation order: argument validation in memory-cache-api/lib.rs runs in the same order as C++.
  • Fallback chain: it follows memory-cache.c++ step for step. Early returns release the permit, so no read is left waiting.
  • Request clock: now_in_current_request() needs no lock, so the Drop impl can safely call it.
  • Tests: each Rust JSG addition has its own test.

⚡ Performance: ✅ LGTM

No performance concerns.


📝 Documentation: ✅ LGTM

The README wording noted earlier is fixed (src/rust/jsg/README.md:252-253).


Reviewed commit: 2393ff7c

…-cache-read

# Conflicts:
#	src/rust/jsg-test/ffi.c++
#	src/rust/jsg-test/ffi.h
#	src/rust/jsg-test/lib.rs
#	src/workerd/api/memory-cache-test.c++
#	src/workerd/util/autogate.h

@clanky-the-clanker clanky-the-clanker Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by the review of bafc2029.

This review, of 0db1504b

✅ Approved — No findings. The changes since the last review (merge conflict resolution and test updates) are clean, and the Rust MemoryCache stays behind an autogate that is off by default.

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


✅ All checks passed

🔒 Security: ✅ LGTM

  • Namespaces: bindForRust() uses the same cache id and limits as the C++ path, so cache isolation is unchanged.
  • DOMException hook: it runs no JavaScript.
  • Rollout: the Rust MemoryCache is behind an autogate that is off by default.

📖 AGENTS.md: ✅ LGTM

No AGENTS.md changes are needed.


📋 Code Quality: ✅ LGTM

  • Test shim: cpp_tunnel_exception and its Rust wrapper handle a failed conversion by returning Error::pending().
  • Span parity test: it checks that C++ records the expected spans, then compares the spans of the two implementations.
  • Cross-request test: it now checks the whole error message, allowing only the C++ suffix.
  • Autogate: workerd-api.c++ creates the Rust MemoryCache only when the autogate is on.

⚡ Performance: ✅ LGTM

No performance concerns.


📝 Documentation: ✅ LGTM

No documentation issues.


Reviewed commit: 0db1504b

bartlomieju and others added 2 commits November 11, 2026 15:52
A span reports to its request's observer, whose state belongs to the
request's thread, so SpanBuilder is not Send, and IoContext::add_object()
takes only Send values. Something whose lifetime the garbage collector
decides, such as the reaction to a promise, can be dropped on whichever
thread next locks the isolate, so it cannot hold a span safely. C++ has
the IoContext own such spans; Rust had no way to.

- Add IoContext::add_span(), which gives the IoContext a span, as
  add_object() does a Send value. The IoContext only ever touches it on
  its own thread: IoOwned::take() and with() require it to be current,
  and it drops the slot there, at once or through its delete queue
  (DeleteQueue::scheduleDeletion()). Dropping the handle elsewhere never
  drops the span.
- Add IoOwned::with(), which uses the value in place, as dereferencing
  an IoOwn does, failing unless its IoContext is current.
- add_object() keeps its Send bound: a non-Send value in general could
  share an std::rc::Rc with something GC drops on another thread.
- Test both, and that the handle to a span cannot go in a Send future.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Rust MemoryCache had the bug that 9e9c7ef fixed in C++: the Fill
that the reactions to a fallback's promise share held the read and
fallback spans. V8 owns those reactions; if the fallback never settles,
it drops them only when it collects them, on whichever thread then holds
the isolate lock, ending the spans there, racing with the request's
thread over its observer's state.

The Fill now gives the request both spans with IoContext::add_span(), so
they end no later than the request does, on its thread. As C++ does, the
reactions reach the spans first, failing in another request before the
value is serialized, which can run JavaScript. The child span is
released before its parent, as in C++.

"the spans of a fallback that never settles end with its request" now
runs against both implementations; without this change, the Rust spans
stay open.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@clanky-the-clanker clanky-the-clanker Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by the review of b5ae6b8b.

This review, of bafc2029

✅ Approved — No findings. The changes since the last review (the IoContext now owns the fallback spans, the new io-context clock and span APIs, and their tests) look sound, and the Rust MemoryCache is still behind an autogate that is off by default.

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


✅ All checks passed

🔒 Security: ✅ LGTM

  • Keys: keys are length-checked in UTF-8 bytes, and non-string keys are rejected.
  • Cross-request access: IoOwned::with and take gate access to the fallback spans and permit, so another request cannot reach them.
  • Cache contents: the cache holds only serialized bytes, so no V8 references cross requests.
  • Namespaces: the binding uses the same namespaces as the C++ path.
  • Rollout: the Rust MemoryCache is behind an autogate that is off by default.

📖 AGENTS.md: ✅ LGTM

No AGENTS.md changes are needed.


📋 Code Quality: ✅ LGTM

  • fallback.rs: Fill::new hands the read span to the request before the fallback span. fail() and succeed() each take the permit exactly once. While execution is terminating, the fallback path returns without throwing.
  • io-context/lib.rs: now_in_current_request() needs no lock and returns None outside a request. IoOwned::with() checks the current request before it touches the value, and a panic cannot poison its mutex. IoOwned<SpanBuilder> stays non-Send, which the new compile test checks.
  • memory-cache-test.c++: recordSpans returns spans in the order they open, and the never-settling-fallback test runs against both implementations.
  • Autogate wiring: workerd-api.c++ creates the Rust MemoryCache only when RUST_MEMORY_CACHE is on. With the autogate off, the C++ path behaves as before.

⚡ Performance: ✅ LGTM

No performance concerns. Allocations per read and the promise reactions per fallback match the C++ implementation.


📝 Documentation: ✅ LGTM

No documentation issues.


Reviewed commit: bafc2029

@clanky-the-clanker clanky-the-clanker Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by the review of b30e0d0b.

This review, of b5ae6b8b

✅ Approved — No findings. The merge from main since the last review left the reviewed patches unchanged, and the Rust MemoryCache is still behind an autogate that is off by default.

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


✅ All checks passed

🔒 Security: ✅ LGTM

  • Keys: keys must be strings and are length-checked in UTF-8 bytes before any cache access.
  • Deserialization: deserialize_with only adds an option that drops error stacks.
  • Cache contents: the cache holds only serialized bytes, so no V8 references cross requests.
  • Rollout: the Rust path is behind an autogate that is off by default.

📖 AGENTS.md: ✅ LGTM

No AGENTS.md changes are needed.


📋 Code Quality: ✅ LGTM

The Rust MemoryCache matches memory-cache.c++ on every path checked:

  • Arguments and keys: key and argument validation, and errors that throw or reject depending on capture_async_api_throws.
  • Key decoding: keys are decoded from UTF-8 bytes.
  • Trace spans: span names and tags match C++.
  • Exceptions: TunneledException follows the same order as exceptionToKj().

Other changes:

  • io-context: now_in_current_request() needs no lock and returns None outside a request. IoOwned::with() checks which request is current before it touches the value.
  • Autogate wiring: in workerd-api.c++, both paths build the limits once and share them, so with the gate off the C++ path behaves as before.

⚡ Performance: ✅ LGTM

Allocations per read and promise reactions per fallback match the C++ implementation. CacheBinding::drop reads the clock without taking a lock.


📝 Documentation: ✅ LGTM

No documentation issues.


Reviewed commit: b5ae6b8b

@clanky-the-clanker clanky-the-clanker Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by the review of 0cc8a97b.

This review, of b30e0d0b

✅ Approved — No findings. The merge from main since the last review left the reviewed patches unchanged, and the Rust MemoryCache is still behind an autogate that is off by default.

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


✅ All checks passed

🔒 Security: ✅ LGTM

  • Keys: keys must be strings and are length-checked in UTF-8 bytes before any cache access.
  • Deserialization: deserialize_with only adds an option that drops error stacks.
  • Cache contents: the cache holds only serialized bytes, so no V8 references cross requests.
  • Rollout: the Rust path is behind an autogate that is off by default.

📖 AGENTS.md: ✅ LGTM

No AGENTS.md changes are needed.


📋 Code Quality: ✅ LGTM

The Rust MemoryCache matches memory-cache.c++ on every path checked:

  • Arguments and keys: key and argument validation, and errors that throw or reject depending on capture_async_api_throws.
  • Key decoding: keys are decoded from UTF-8 bytes.
  • Trace spans: span names and tags match C++.
  • Exceptions: TunneledException follows the same order as exceptionToKj().

Other changes:

  • io-context: now_in_current_request() needs no lock and returns None outside a request. IoOwned::with() checks which request is current before it touches the value.
  • Autogate wiring: in workerd-api.c++, both paths build the limits once and share them, so with the gate off the C++ path behaves as before.

⚡ Performance: ✅ LGTM

Allocations per read and promise reactions per fallback match the C++ implementation. CacheBinding::drop reads the clock without taking a lock.


📝 Documentation: ✅ LGTM

No documentation issues.


Reviewed commit: b30e0d0b — same changes as b5ae6b8b, so this is that commit's review (no new review was run; line numbers are as of b5ae6b8b)

C++ JSG names the type in "Cannot initialize ... from an undefined or
null value." as typeid() does, which is mangled on Linux and macOS but
"struct workerd::api::CacheValueProduceResult" on Windows. The space
broke fallbackProducesNoObject's regex there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@clanky-the-clanker clanky-the-clanker Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by the review of e1cb7156.

This review, of 0cc8a97b

✅ Approved — No findings. The only change since the last review is a test file. It is consistent with the C++ and Rust implementations, and the Rust path is still behind an autogate that is off by default.

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


✅ All checks passed

🔒 Security: ✅ LGTM

  • Keys: keys are validated and length-checked in UTF-8 bytes before any cache access.
  • Deserialization: deserialize_with only adds an option that drops error stacks.
  • Thread safety: the unsafe impl Send for ThreadBoundSlotRef is consistent with its SAFETY comment.
  • Rollout: the Rust path is behind an autogate that is off by default.

📖 AGENTS.md: ✅ LGTM

No AGENTS.md changes are needed.


📋 Code Quality: ✅ LGTM

The updated memory-cache-test.js is consistent with memory-cache.c++ and the Rust port.

  • The cross-request regex accepts the optional C++ (I/O type: …FallbackSpans…) suffix.
  • The delete() request requirement matches C++.
  • The new tests use only bindings that the .wd-test config defines.

The io-context, jsg, server wiring and autogate changes are unchanged and still correct.


⚡ Performance: ✅ LGTM

The only change is to tests, and it brings no performance regression. The timed waits in the race tests are cancelled once the other side settles.


📝 Documentation: ✅ LGTM

No documentation issues.


Reviewed commit: 0cc8a97b

…-cache-read

Main now requires Send for Traced (1effdc1), so the data of a promise's
reaction must be Send. Adapt the Rust MemoryCache's fallback to it:

- IoContext::add_span() returns an IoOwnedSpan, which is Send and Sync,
  rather than an IoOwned<SpanBuilder>, which is neither. It reaches the
  span only through IoOwned::take() and with(), on the IoContext's
  thread; the span is kept in a private wrapper declared Send, with the
  same reasoning as the ThreadBoundSlotRef it replaces.
- io-context's IoOwnedObject is declared Sync: Rust has no method on it.
- SharedFill holds an Arc<Fill> rather than an std::rc::Rc.
- Replace the compile test that the handle to a span is not Send with
  one that the span taken out of it is not, and one that the handle can
  be held in Traced data.

Conflicts:
	src/rust/jsg/v8.rs

@clanky-the-clanker clanky-the-clanker Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Approved — No findings. The span-ownership and Send/Sync changes since the last review are sound and tested, and the Rust MemoryCache is still behind an autogate that is off by default.

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


✅ All checks passed

🔒 Security: ✅ LGTM

  • unsafe impl Send for ThreadBoundSpan: sound. Every way of reaching the span checks for the owning request first.
  • unsafe impl Sync for the opaque IoOwnedObject: acceptable, since Rust has no methods on the type.
  • Keys: they are decoded from UTF-8 and length-checked before any cache access.
  • Rollout: the Rust path is behind an autogate that is off by default.

📖 AGENTS.md: ✅ LGTM

No AGENTS.md changes are needed.


📋 Code Quality: ✅ LGTM

  • IoOwnedSpan and IoContext span ownership:
    • Fill holds only Send handles.
    • The parent span is added before the child, so the child ends first.
    • When the current request is a different one, every way of reaching a span fails as C++ does.
  • FFI declarations: the jsg declarations in ffi.h and v8.rs match their C++ shims and respect the pending-exception rules.
  • now_in_current_request(): it is covered for both the fixed-clock case and the case with no request.

⚡ Performance: ✅ LGTM

No performance issues.


📝 Documentation: ✅ LGTM

No documentation issues.


Reviewed commit: e1cb7156

@bartlomieju
bartlomieju merged commit 03897a4 into main Nov 12, 2026
36 of 37 checks passed
@bartlomieju
bartlomieju deleted the biwanczuk/rust-memory-cache-read branch November 12, 2026 11:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants