Repository navigation
Run the workers-sdk test suites concurrently - #8352
Conversation
The workers-sdk-test job ran five suites one after another, and each suite's turbo run first rebuilt its dependencies from scratch. The job took about 12.5 minutes, mostly with idle cores. Build every package the suites need in one turbo run, then run the five suites at the same time with `--only`, so none rebuilds what is already built. The Vite and Vitest suites run 4 workers each instead of their defaults: at full worker counts, a Vitest fixture test that waits 10s for 100 hibernatable WebSocket messages timed out in 2 of 3 runs. Each suite's output is printed in its own log group once all have finished. Miniflare keeps whatever worker count workers-sdk configures, so this doesn't depend on any workers-sdk change. It currently runs one test file at a time and takes about 6 minutes, which makes it the longest suite. cloudflare/workers-sdk#16823 parallelizes it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
e4db63f to
6ba884c
Compare
There was a problem hiding this comment.
Superseded by the review of
307c09a7.
This review, of 6ba884cc
💬 Approved with comments — The CI workflow change is correct for passing and failing runs. One optional suggestion: logs are lost if the job is cancelled or a suite hangs.
Findings: 1 suggestion (1 is an inline comment on the diff)
The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.
📋 Code Quality: 💬 1 suggestion
The concurrent runner is correct. Its output is lost if the step is interrupted before every suite finishes.
Recommendation: 💬 Optional Nits
- [SUGGESTION] Suite logs are printed only after every suite finishes, so a hang or a cancelled run shows no test output (
.github/workflows/test.yml:357)
View full review
.github/workflows/test.yml:357: each suite writes to$RUNNER_TEMP/<name>.log, and the logs are printed only afterwaitreturns. The job has notimeout-minutes, so the default 6 hours applies. If a suite hangs, or the run is cancelled, nothing is printed, including the output of suites that already failed. The serial steps this replaces streamed their output live. Adding a follow-up step withif: cancelled() || failure()that prints$RUNNER_TEMP/*.logwould keep those logs. A step-leveltimeout-minuteswould also stop a hang from holding a runner for hours.
✅ All other checks passed
📝 Documentation: ✅ LGTM
The new workflow comments describe what the steps do and why.
Reviewed commit: 6ba884cc
The concurrent test step prints each suite's log only once every suite has finished, so a hang or a cancelled run showed no test output. Cap the step at 30 minutes, and add a follow-up step that runs on failure or cancellation and prints the log of every suite the main step didn't print, with its exit code when the suite had finished. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
✅ Approved — The 30-minute step timeout and the new log-printing step fix the earlier suggestion about lost logs, and I found no new issues in the workflow change.
The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.
✅ All checks passed
📋 Code Quality: ✅ LGTM
Checked the fix in .github/workflows/test.yml:
- Timeout:
timeout-minutes: 30is set on the test step (line 341). - Logs on disk: each suite writes its output to
$RUNNER_TEMP/<name>.logwhile it runs, so partial output survives a timeout or a cancellation. - Finished vs. unfinished: the
<name>.resultfile is written only after a suite exits. The follow-up step (lines 377-390) uses it to show either the exit code and duration or "did not finish". - No duplicate output: the
.printedmarkers mean a suite printed by the main step is not printed again. bash -e: the default shell does not break the script. Each suite's exit code is captured with|| rc=$?, and the overall result is reported throughexit $failed.
📝 Documentation: ✅ LGTM
The new workflow comments describe what the steps do and why.
Reviewed commit: 307c09a7
The
Run workers-sdk testsjob has grown to about 12.5 minutes, e.g. https://github.com/cloudflare/workerd/actions/runs/42751423815/job/129619178373. It ran five suites one after another, and each suite's turbo run first rebuilt its dependencies from scratch (about 170s of builds in total).This PR:
turbo buildcovers every package the suites need.--only, so none of them rebuilds. Each suite's log is printed in its own group after all have finished, with its exit code and duration. The step fails if any suite fails.preserves hibernatable WebSocket message orderin the Vitest fixtures timed out in 2 of 3 runs. It waits 10s for 100 echoed messages. With full CPU load I confirmed that every message arrives, only slowly, so this is starvation, not a lost message.Miniflare keeps the worker count workers-sdk configures, so this doesn't depend on any workers-sdk change. Today that's one test file at a time, about 6 minutes, which makes Miniflare the longest suite and the job's lower bound. cloudflare/workers-sdk#16823 parallelizes Miniflare's tests and the unenv-preset e2e file, and this job picks that up automatically once it merges.
Timings on a 32-core Linux machine, limited with
tasksetto 16 cores to matchubuntu-24.04-16core, with no turbo cache:main🤖 Generated with Claude Code