Skip to content

Speed up node:buffer transcode() with simdutf and drop its ICU use - #8721

Open
littledivy wants to merge 3 commits into
mainfrom
littledivy/2026-11-20-transcode-simdutf
Open

littledivy wants to merge 3 commits into
mainfrom
littledivy/2026-11-20-transcode-simdutf

Conversation

@littledivy

@littledivy littledivy commented Nov 20, 2026 •

Copy link
Copy Markdown
Member

Every transcode() pair with ASCII or Latin-1 on either side, apart from widening into UTF-16LE, decodes and re-encodes one character at a time, about 3.7 ns per byte, even for ASCII text, which is the same bytes in every encoding transcode() supports. The UTF-8 and UTF-16LE identity pairs still go through ICU's ucnv_convertEx, and they allocate two or three bytes per source byte.

In this patch Latin-1 to UTF-8 uses simdutf's convert_latin1_to_utf8. The other pairs copy the leading bytes that are already valid in the target, using simdutf's validators: ASCII for the pairs with ASCII or Latin-1 on either side, and well-formed text for the identity pairs. Only the bytes after that prefix go through the per-character loop, which now also writes UTF-16LE and replaces ill-formed input as ICU does. With the identity pairs off ICU, src/rust/i18n no longer depends on rust_icu_sys. The first commit adds buffer-transcode-bench.

The bytes written don't change. Differential runs against Node 26 (2240 hand-picked calls, 4000 seeded random calls and 8000 more aimed at the identity pairs and ASCII prefixes) give the same results as before. A well-formed identity source now gets a buffer of exactly its length, so the result's buffer.byteLength is smaller.

Median time per call, optimized build on an Apple M5 Pro:

case before after speedup
ascii->utf8, ASCII, 64 B 468 ns 261 ns 1.79x
ascii->utf8, ASCII, 1 KiB 3.92 µs 279 ns 14.0x
ascii->utf8, ASCII, 64 KiB 237.5 µs 1.91 µs 124x
utf8->latin1, ASCII, 64 KiB 252.6 µs 1.94 µs 130x
latin1->ascii, ASCII, 64 KiB 238.2 µs 1.89 µs 126x
latin1->utf8, ASCII, 64 KiB 239.9 µs 3.66 µs 65.5x
latin1->utf8, é, 64 B 325 ns 271 ns 1.20x
latin1->utf8, é, 64 KiB 117.4 µs 10.5 µs 11.2x
utf8->utf8, ASCII, 64 KiB 20.3 µs 1.99 µs 10.2x
utf8->utf8, €, 64 KiB 52.3 µs 19.0 µs 2.75x
ucs2->ucs2, ASCII, 1 KiB 1.31 µs 364 ns 3.59x
ucs2->ucs2, ASCII, 64 KiB 51.7 µs 4.22 µs 12.3x

@littledivy
littledivy requested review from a team as code owners November 20, 2026 06:41

@clanky-the-clanker clanky-the-clanker Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by the review of e8c56480.

This review, of 78169ec9

💬 Approved with comments — The transcode fast paths, destination sizing and replacement of ill-formed input all trace correctly, and removing rust_icu_sys is clean. The only finding is an INFO about a misleading doc comment on ascii_prefix_len.

Findings: 1 info (1 is an inline comment on the diff)

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


📋 Code Quality: 💬 1 info

Correct dispatch, sizing and replacement logic; one misleading doc comment.

Recommendation: 💬 Optional Nits

  • [INFO] The ascii_prefix_len doc comment says simdutf stops at a 64-byte block, but the contract and its test rely on the exact position of the first non-ASCII byte. (src/rust/i18n/ffi.rs:151)
View full review

I traced the dispatch arms, destination sizing, the copied prefix and the replacement of ill-formed input by hand, and they hold up.

  • Dispatch order: Latin-1 to Latin-1 and Latin-1 to UTF-8 are matched before the prefix arm.
  • Odd-length UTF-16LE: odd-length UTF-16LE input to ASCII or Latin-1 is still rejected.
  • Destination sizing: the worst-case size of the remainder is added to the copied prefix.
  • Latin-1 to UTF-8: convert_to_utf8 checks the target length before the unsafe call.
  • Dependency removal: nothing still references rust_icu_sys or the ICU version pin.

The one finding is a doc comment. It says ascii_prefix_len stops at a 64-byte block, but ffi.h and ascii_prefix_stops_at_the_first_non_ascii_byte both rely on it returning the exact position of the first non-ASCII byte.


✅ All other checks passed

🔒 Security: ✅ LGTM

No security issues found.

  • Destination sizing: each destination buffer is sized from the same source that is later converted.
  • Latin-1 to UTF-8: the simdutf call is preceded by a check of the target length.
  • Odd-length UTF-16LE: for UTF-16LE identity input of odd length, (len + 1) & !1 leaves room for the U+FFFD written for the trailing odd byte.

📖 AGENTS.md: ✅ LGTM

No AGENTS.md updates needed.


⚡ Performance: ✅ LGTM

No performance regressions.

  • Fast path: for valid input, the validator prefix handles everything in one pass and the destination is exactly the source's length.
  • Scalar fallback: it is O(n) over the remainder only, and is still bounded by ISOLATE_LIMIT.

📝 Documentation: ✅ LGTM

The changes to docs/v8-updates.md match the removal of the ICU version pin from ci/v8_update.py.


Reviewed commit: 78169ec9

Comment thread src/rust/i18n/ffi.rs Resolved
There was no benchmark for `transcode()`, so changes to its conversion paths couldn't be measured.

In this patch `buffer-transcode-bench` times `transcode()` for 14 encoding pairs, covering every conversion path, on ASCII and non-ASCII text of 64 bytes, 1 KiB and 64 KiB.
Every `transcode()` pair with ASCII or Latin-1 on either side, apart from widening into UTF-16LE, decodes and re-encodes one character at a time, about 3.7 ns per byte. That holds even for ASCII text, which is the same bytes in every encoding `transcode()` supports.

In this patch Latin-1 to UTF-8 uses simdutf's `utf8_length_from_latin1` and `convert_latin1_to_utf8`, and Latin-1 to Latin-1 is a copy. For the other pairs into ASCII or Latin-1, and ASCII to UTF-8, simdutf's `validate_ascii_with_errors` finds where the leading ASCII bytes end. An all-ASCII source is copied, and otherwise the ASCII prefix is copied and only the rest goes through the per-character loop, so text with non-ASCII bytes is no slower. The output doesn't change.

Median time per call, `buffer-transcode-bench`, optimized build on an Apple M5 Pro:

| case | before | after | speedup |
|---|---:|---:|---:|
| ascii->utf8, ASCII, 1 KiB | 3.92 µs | 279 ns | 14.0x |
| ascii->utf8, ASCII, 64 KiB | 237.5 µs | 1.91 µs | 124x |
| utf8->latin1, ASCII, 64 KiB | 252.6 µs | 1.94 µs | 130x |
| latin1->ascii, ASCII, 64 KiB | 238.2 µs | 1.89 µs | 126x |
| latin1->utf8, ASCII, 64 KiB | 239.9 µs | 3.66 µs | 65.5x |
| latin1->utf8, é, 64 B | 325 ns | 271 ns | 1.20x |
| latin1->utf8, é, 64 KiB | 117.4 µs | 10.5 µs | 11.2x |
@littledivy
littledivy force-pushed the littledivy/2026-11-20-transcode-simdutf branch from 78169ec to e8c5648 Compare November 20, 2026 06:47

@clanky-the-clanker clanky-the-clanker Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by the review of 8fda1a70.

This review, of e8c56480

✅ Approved — The change to ffi.rs since the last review fixes the ascii_prefix_len doc comment, and nothing else in it raises a concern. No findings remain.

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


✅ All checks passed

🔒 Security: ✅ LGTM

  • simdutf wrappers: each unsafe wrapper passes a pointer and length taken from a live slice.
  • Latin-1 to UTF-8: Latin1Source::convert_to_utf8 checks the target length before converting, so the destination cannot overflow.
  • UTF-16LE prefix: utf16le_valid_prefix_len works on whole code units only, so it never reads past the slice.

📖 AGENTS.md: ✅ LGTM

No AGENTS.md updates needed.


📋 Code Quality: ✅ LGTM

The ascii_prefix_len doc comment now says the function returns the exact position of the first non-ASCII byte. That matches ffi.h and the test that checks it.

  • Prefix wrappers: the UTF-8 and UTF-16LE wrappers convert code units to bytes correctly, and a trailing odd byte is excluded.
  • Latin-1 to UTF-8: Latin1Source::convert_to_utf8 checks the target length before the unsafe call.

⚡ Performance: ✅ LGTM

The simdutf wrappers each make one linear pass over a slice and allocate nothing per call. The scalar fallback runs only on the bytes after the validated prefix.


📝 Documentation: ✅ LGTM

The ascii_prefix_len doc comment now describes its contract correctly. The changes to docs/v8-updates.md match the removal of the ICU version pin from ci/v8_update.py.


Reviewed commit: e8c56480

`transcode()` from UTF-8 to UTF-8 and from UTF-16LE to UTF-16LE is the last use of ICU in `src/rust/i18n`. It runs every byte through `ucnv_convertEx` and allocates three (UTF-8) or two (UTF-16LE) bytes per source byte, though nearly all inputs are already well formed.

In this patch simdutf's validators find how much of the source is well formed. That prefix is copied, and only the bytes from the first ill-formed sequence on go through the per-character loop that already handles the ASCII and Latin-1 pairs, which now also writes UTF-16LE. It writes U+FFFD as ICU does: one per maximal subpart of ill-formed UTF-8 and one per unpaired surrogate, and one for a trailing odd byte together with a lead surrogate before it. `single_byte.rs` becomes `scalar.rs`, since it now covers more than single-byte pairs. `Converter`, `convert_ex` and the crate's `rust_icu_sys` dependency go, and with them the ICU version pin in its `BUILD.bazel` that `ci/v8_update.py` kept in step.

The bytes written don't change; differential runs against Node 26 give the same results as before. A well-formed source now gets a buffer of exactly its length, so the result's `buffer.byteLength` is smaller, and well-formed identity inputs over 43 MiB (UTF-8) or 64 MiB (UTF-16LE) no longer fail the 128 MiB size check.

| case | before | after | speedup |
|---|---:|---:|---:|
| utf8->utf8, ASCII, 64 KiB | 20.3 µs | 1.99 µs | 10.2x |
| utf8->utf8, €, 1 KiB | 1.39 µs | 579 ns | 2.39x |
| utf8->utf8, €, 64 KiB | 52.3 µs | 19.0 µs | 2.75x |
| ucs2->ucs2, ASCII, 1 KiB | 1.31 µs | 364 ns | 3.59x |
| ucs2->ucs2, ASCII, 64 KiB | 51.7 µs | 4.22 µs | 12.3x |
@littledivy
littledivy force-pushed the littledivy/2026-11-20-transcode-simdutf branch from e8c5648 to 8fda1a7 Compare November 20, 2026 07:19
@littledivy littledivy changed the title Speed up node:buffer transcode() with simdutf and drop ICU Speed up node:buffer transcode() with simdutf and drop its ICU use Nov 20, 2026

@clanky-the-clanker clanky-the-clanker Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💬 Approved with comments — No correctness or safety problems. The only note is an INFO: the rust_icu_sys dependency is still declared and pinned even though nothing uses it.

Findings: 1 info (1 is an inline comment on the diff)

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


📋 Code Quality: 💬 1 info

The pin-list and docs changes are consistent. One INFO: rust_icu_sys is still declared and pinned, but nothing uses it.

Recommendation: 💬 Optional Nits

  • [INFO] rust_icu_sys is still declared (deps/rust/Cargo.toml:88), annotated (build/deps/rust.MODULE.bazel:105) and pinned, though nothing depends on it after this change. (ci/v8_update.py:41)
View full review
  • rust_icu_sys has no users: src/rust/i18n/BUILD.bazel no longer depends on it, and no other BUILD or Rust file in the repository uses it. It is still declared in deps/rust/Cargo.toml:88 and annotated in build/deps/rust.MODULE.bazel:105, and ci/v8_update.py still keeps its ICU version pin up to date.
  • Checked, no issues: the UTF-16LE prefix leaves out a trailing odd byte, the scalar path emits a single U+FFFD for a truncated code unit, and the destination sizing leaves room for the copied prefix.

✅ All other checks passed

🔒 Security: ✅ LGTM

No security concerns in the changes since the last review.


📖 AGENTS.md: ✅ LGTM

No AGENTS.md updates needed.


⚡ Performance: ✅ LGTM

No performance concerns.


📝 Documentation: ✅ LGTM

docs/v8-updates.md matches the updated pin list in ci/v8_update.py.


Reviewed commit: 8fda1a70

Comment thread ci/v8_update.py
ROOT / "build/deps/rust.MODULE.bazel",
ROOT / "src/rust/i18n/BUILD.bazel",
)
RUST_ICU_VERSION_PINS = (ROOT / "build/deps/rust.MODULE.bazel",)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ INFO · Code Quality

rust_icu_sys is still declared (deps/rust/Cargo.toml:88), annotated (build/deps/rust.MODULE.bazel:105) and pinned, though nothing depends on it after this change.

After this change nothing in the repository depends on rust_icu_sys: src/rust/i18n/BUILD.bazel drops it, and no other BUILD or Rust file uses it. The crate is still declared in deps/rust/Cargo.toml and annotated in build/deps/rust.MODULE.bazel. As a result, every V8 update still has to bump its ICU pin and check that bindgen/lib_<version>.rs exists, for a crate that is never built. Consider removing the crate, the annotation, this pin, _update_rust_icu_version() and the docs step, here or in a follow-up. If it is kept on purpose, a note saying why would help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant