Skip to content

Run the workers-sdk test suites concurrently - #8352

Merged
danlapid merged 2 commits into
mainfrom
dlapid/workers-sdk-test-concurrency
Nov 11, 2026
Merged

danlapid merged 2 commits into
mainfrom
dlapid/workers-sdk-test-concurrency

Conversation

@danlapid

@danlapid danlapid commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

The Run workers-sdk tests job has grown to about 12.5 minutes, e.g. https://github.com/cloudflare/workerd/actions/runs/42751423815/job/129619178373. It ran five suites one after another, and each suite's turbo run first rebuilt its dependencies from scratch (about 170s of builds in total).

This PR:

  • Builds once. A single turbo build covers every package the suites need.
  • Runs the suites concurrently with --only, so none of them rebuilds. Each suite's log is printed in its own group after all have finished, with its exit code and duration. The step fails if any suite fails.
  • Caps Vite and Vitest. Those two suites run 4 workers each instead of their defaults. At full worker counts, preserves hibernatable WebSocket message order in the Vitest fixtures timed out in 2 of 3 runs. It waits 10s for 100 echoed messages. With full CPU load I confirmed that every message arrives, only slowly, so this is starvation, not a lost message.

Miniflare keeps the worker count workers-sdk configures, so this doesn't depend on any workers-sdk change. Today that's one test file at a time, about 6 minutes, which makes Miniflare the longest suite and the job's lower bound. cloudflare/workers-sdk#16823 parallelizes Miniflare's tests and the unenv-preset e2e file, and this job picks that up automatically once it merges.

Timings on a 32-core Linux machine, limited with taskset to 16 cores to match ubuntu-24.04-16core, with no turbo cache:

Setup Time for the test steps
Today's job (CI run above) ~11.5 min
This PR with workers-sdk main expected ~84s build + ~6 min (Miniflare, serial); this PR's own CI run measures it
This PR with workers-sdk#16823 84s build + 143–145s tests

🤖 Generated with Claude Code

The workers-sdk-test job ran five suites one after another, and each
suite's turbo run first rebuilt its dependencies from scratch. The
job took about 12.5 minutes, mostly with idle cores.

Build every package the suites need in one turbo run, then run the
five suites at the same time with `--only`, so none rebuilds what is
already built. The Vite and Vitest suites run 4 workers each instead
of their defaults: at full worker counts, a Vitest fixture test that
waits 10s for 100 hibernatable WebSocket messages timed out in 2 of
3 runs. Each suite's output is printed in its own log group once all
have finished.

Miniflare keeps whatever worker count workers-sdk configures, so this
doesn't depend on any workers-sdk change. It currently runs one test
file at a time and takes about 6 minutes, which makes it the longest
suite. cloudflare/workers-sdk#16823 parallelizes it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@danlapid
danlapid force-pushed the dlapid/workers-sdk-test-concurrency branch from e4db63f to 6ba884c Compare November 11, 2026 02:23
@danlapid
danlapid marked this pull request as ready for review November 11, 2026 02:31
@danlapid
danlapid requested review from a team as code owners November 11, 2026 02:31

@clanky-the-clanker clanky-the-clanker Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by the review of 307c09a7.

This review, of 6ba884cc

💬 Approved with comments — The CI workflow change is correct for passing and failing runs. One optional suggestion: logs are lost if the job is cancelled or a suite hangs.

Findings: 1 suggestion (1 is an inline comment on the diff)

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


📋 Code Quality: 💬 1 suggestion

The concurrent runner is correct. Its output is lost if the step is interrupted before every suite finishes.

Recommendation: 💬 Optional Nits

  • [SUGGESTION] Suite logs are printed only after every suite finishes, so a hang or a cancelled run shows no test output (.github/workflows/test.yml:357)
View full review
  • .github/workflows/test.yml:357: each suite writes to $RUNNER_TEMP/<name>.log, and the logs are printed only after wait returns. The job has no timeout-minutes, so the default 6 hours applies. If a suite hangs, or the run is cancelled, nothing is printed, including the output of suites that already failed. The serial steps this replaces streamed their output live. Adding a follow-up step with if: cancelled() || failure() that prints $RUNNER_TEMP/*.log would keep those logs. A step-level timeout-minutes would also stop a hang from holding a runner for hours.

✅ All other checks passed

📝 Documentation: ✅ LGTM

The new workflow comments describe what the steps do and why.


Reviewed commit: 6ba884cc

Comment thread .github/workflows/test.yml Resolved
The concurrent test step prints each suite's log only once every suite
has finished, so a hang or a cancelled run showed no test output. Cap
the step at 30 minutes, and add a follow-up step that runs on failure
or cancellation and prints the log of every suite the main step didn't
print, with its exit code when the suite had finished.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@clanky-the-clanker clanky-the-clanker Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Approved — The 30-minute step timeout and the new log-printing step fix the earlier suggestion about lost logs, and I found no new issues in the workflow change.

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


✅ All checks passed

📋 Code Quality: ✅ LGTM

Checked the fix in .github/workflows/test.yml:

  • Timeout: timeout-minutes: 30 is set on the test step (line 341).
  • Logs on disk: each suite writes its output to $RUNNER_TEMP/<name>.log while it runs, so partial output survives a timeout or a cancellation.
  • Finished vs. unfinished: the <name>.result file is written only after a suite exits. The follow-up step (lines 377-390) uses it to show either the exit code and duration or "did not finish".
  • No duplicate output: the .printed markers mean a suite printed by the main step is not printed again.
  • bash -e: the default shell does not break the script. Each suite's exit code is captured with || rc=$?, and the overall result is reported through exit $failed.

📝 Documentation: ✅ LGTM

The new workflow comments describe what the steps do and why.


Reviewed commit: 307c09a7

Comment thread .github/workflows/test.yml Resolved
@danlapid
danlapid merged commit 1b8b7f6 into main Nov 11, 2026
23 checks passed
@danlapid
danlapid deleted the dlapid/workers-sdk-test-concurrency branch November 11, 2026 03:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants