Skip to content

Relocate files that are used for the Pyodide bundle - #8229

Merged
ryanking13 merged 1 commit into
mainfrom
gyeongjae/pyodide-bundle-dir
Nov 13, 2026
Merged

ryanking13 merged 1 commit into
mainfrom
gyeongjae/pyodide-bundle-dir

Conversation

@ryanking13

@ryanking13 ryanking13 commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

No functional change included.

Relocates files that are used to build Pyodide bundle. Previously files that constructs Pyodide bundle were here and there mixed in different directories, so it was confusing for both humans (mostly me) and AI agents to decide whether we need to update the Pyodide bundle or not.

This PR puts most of the files (except for python_metadata.bzl) that are related to the Pyodide bundle creation in to the bundle subdirectory, so that we can easily know whether the new bundle needs to be created or not.

I'll open a few more follow-up PRs to make things cleaner.

@ryanking13
ryanking13 requested review from a team as code owners November 5, 2026 07:21

@clanky-the-clanker clanky-the-clanker Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by the review of baa4ad88.

This review, of 62ff8923

💬 Approved with comments — A clean relocation with consistent references. The only note is an INFO: the emscripten_setup_override parameter was dropped during the move and the description doesn't mention it.

Findings: 1 info (1 is an inline comment on the diff)

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


📋 Code Quality: 💬 1 info

The relocation is consistent. One undisclosed parameter removal.

Recommendation: 💬 Optional Nits

  • [INFO] emscripten_setup_override was dropped from _python_bundle during the move, although the description says there is no functional change (src/pyodide/bundle/python_bundles.bzl:61)
View full review

The moved files, labels, exports_files and visibility all line up, and nothing still references pyodide/tools, the old python-lock location or src/pyodide/python_packages. The pyodide-internal:generated/pyodide.asm.wasm module declaration removed from pyodide.asm.d.ts has no importer.

One note: the moved _python_bundle drops the emscripten_setup_override parameter, along with its copy-instead-of-esbuild branch. That branch existed in helpers.bzl at the base revision. Nothing in this repository passes the parameter, but python_bundles(overrides = ...) is a macro other repositories can call.


✅ All other checks passed

🔒 Security: ✅ LGTM

This is a build-file relocation. It adds no secrets, injection paths or access-control changes.


📖 AGENTS.md: ✅ LGTM

The AGENTS.md updates in build/ and src/pyodide/ match the new file locations.


⚡ Performance: ✅ LGTM

No performance impact. The change only moves build files.


📝 Documentation: ✅ LGTM

The documentation path references were updated to match the new layout.


Reviewed commit: 62ff8923

Comment thread src/pyodide/bundle/python_bundles.bzl Resolved
@ryanking13
ryanking13 force-pushed the gyeongjae/pyodide-bundle-dir branch from 62ff892 to baa4ad8 Compare November 9, 2026 11:04

@clanky-the-clanker clanky-the-clanker Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by the review of 6b3b7472.

This review, of baa4ad88

💬 Approved with comments — A clean relocation with consistent references. The only note is an INFO: the emscripten_setup_override parameter was dropped during the move and the description doesn't mention it.

Findings: 1 info

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


📋 Code Quality: 💬 1 info

The relocation is consistent. One undisclosed parameter removal.

Recommendation: 💬 Optional Nits

  • [INFO] emscripten_setup_override was dropped from _python_bundle during the move, although the description says there is no functional change (src/pyodide/bundle/python_bundles.bzl:61)
View full review

The moved files, labels, exports_files and visibility all line up, and nothing still references pyodide/tools, the old python-lock location or src/pyodide/python_packages. The pyodide-internal:generated/pyodide.asm.wasm module declaration removed from pyodide.asm.d.ts has no importer.

One note: the moved _python_bundle drops the emscripten_setup_override parameter, along with its copy-instead-of-esbuild branch. That branch existed in helpers.bzl at the base revision. Nothing in this repository passes the parameter, but python_bundles(overrides = ...) is a macro other repositories can call.


✅ All other checks passed

🔒 Security: ✅ LGTM

This is a build-file relocation. It adds no secrets, injection paths or access-control changes.


📖 AGENTS.md: ✅ LGTM

The AGENTS.md updates in build/ and src/pyodide/ match the new file locations.


⚡ Performance: ✅ LGTM

No performance impact. The change only moves build files.


📝 Documentation: ✅ LGTM

The documentation path references were updated to match the new layout.


Reviewed commit: baa4ad88 — same changes as 62ff8923, so this is that commit's review (no new review was run; line numbers are as of 62ff8923)

@hoodmane

hoodmane commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Isn't everything in the Pyodide directory either code in the bundle or stuff to make the bundle? What doesn't fall into one of these categories? Tools?

@ryanking13

ryanking13 commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor Author

Isn't everything in the Pyodide directory either code in the bundle or stuff to make the bundle? What doesn't fall into one of these categories? Tools?

@hoodmane I want to split the ones that go into Pyodide bundle (that requires a new bundle release) and ones that are linked to workerd binary. In a follow-up, I am preparing for a PR to reduce the files that goes into Pyodide bundle to make it smaller + less frequent Pyodide bundle release.

@hoodmane

Copy link
Copy Markdown
Contributor

Okay that makes sense.

@hoodmane

Copy link
Copy Markdown
Contributor

I'm still confused here, isn't everything in internal also in the bundle?

@ryanking13

ryanking13 commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor Author

I'm still confused here, isn't everything in internal also in the bundle?

Yes, but not all. Most of them are linked to the pyodide_static module that goes to the workerd binary in the end. Only some of them emscriptenSetup.ts and utility files go into the bundle. I'll split them properly as a follow-up. It will need a bundle update so I didn't to in this PR.

@hoodmane

Copy link
Copy Markdown
Contributor

But don't a bunch of the utility files go into both the workerd binary and the bundle?

@ryanking13
ryanking13 force-pushed the gyeongjae/pyodide-bundle-dir branch from baa4ad8 to 6b3b747 Compare November 11, 2026 03:22

@clanky-the-clanker clanky-the-clanker Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by the review of fff014d4.

This review, of 6b3b7472

✅ Approved — A clean relocation of the bundle build files with consistent label and path updates; nothing is left to fix.

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


✅ All checks passed

🔒 Security: ✅ LGTM

This is a build-file relocation. It adds no secrets, injection paths or access-control changes.


📖 AGENTS.md: ✅ LGTM

The AGENTS.md updates in build/ and src/pyodide/ match the new file locations.


📋 Code Quality: ✅ LGTM

The updated justfile eslint recipe points at //src/pyodide/bundle:patch_pyodide_asm_lib@eslint, which src/pyodide/bundle/BUILD.bazel defines. Nothing still refers to the old paths. The note about the dropped emscripten_setup_override parameter is closed: nothing in the repository uses it.


⚡ Performance: ✅ LGTM

No performance impact. The change only moves build files and updates paths.


📝 Documentation: ✅ LGTM

The documentation path references match the new layout.


Reviewed commit: 6b3b7472

@ryanking13

Copy link
Copy Markdown
Contributor Author

But don't a bunch of the utility files go into both the workerd binary and the bundle?

That is what I am going to drop in a follow-up PR. Here is a stacked PR for it: #8360

@ryanking13
ryanking13 force-pushed the gyeongjae/pyodide-bundle-dir branch from 6b3b747 to fff014d Compare November 13, 2026 03:07

@clanky-the-clanker clanky-the-clanker Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by the review of 69ae8766.

This review, of fff014d4

✅ Approved — This is a clean relocation of the Pyodide bundle build files: labels, include paths and lock-file paths are all updated consistently, and nothing is left to fix.

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


✅ All checks passed

🔒 Security: ✅ LGTM

The change only moves build files. It adds no secrets, injection paths or access-control changes.


📖 AGENTS.md: ✅ LGTM

The AGENTS.md updates in build/ and src/pyodide/ match the new file locations.


📋 Code Quality: ✅ LGTM

  • python_bundles.bzl uses Label(":..."), so its labels resolve to //src/pyodide/bundle. Inputs that belong to the calling package (internal/pool/*.ts, internal/util.ts, internal/const.ts) stay as plain strings, which is correct.
  • The python_packages_capnp target, its dependency in src/workerd/api/BUILD.bazel and the include in pyodide.h all agree with each other.
  • MODULE.bazel.lock and dep_pyodide.bzl use the same lock-file paths.
  • The extends path in the renamed bundle/tsconfig.json still resolves at its new location.
  • src/pyodide/types/pyodide.asm.d.ts drops the pyodide-internal:generated/pyodide.asm.wasm module declaration. Nothing imports that module, so the removal changes no behaviour.

⚡ Performance: ✅ LGTM

The change has no performance impact: it only moves files and updates paths.


📝 Documentation: ✅ LGTM

The path references in the documentation match the new layout.


Reviewed commit: fff014d4

@ryanking13
ryanking13 force-pushed the gyeongjae/pyodide-bundle-dir branch from fff014d to 69ae876 Compare November 13, 2026 04:45

@clanky-the-clanker clanky-the-clanker Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Approved — A clean relocation of the Pyodide bundle build files with all paths and labels updated consistently; the one change since the last review (a comment in loadPackage.ts) adds no issues.

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


✅ All checks passed

🔒 Security: ✅ LGTM

The change only moves build files and updates paths. It adds no secrets, injection paths or access-control changes.


📖 AGENTS.md: ✅ LGTM

The AGENTS.md updates in build/ and src/pyodide/ match the new file locations.


📋 Code Quality: ✅ LGTM

  • python_bundles.bzl uses Label(":..."), so its labels resolve to //src/pyodide/bundle. Inputs that belong to the calling package stay as plain strings, which is correct.
  • Both tools keep //src/pyodide:__pkg__ visibility, which is enough for the rules that call them.
  • After simplification, _out_path, _ts_bundle_out and _copy_to_generated still produce the same output names as before.
  • MODULE.bazel.lock and dep_pyodide.bzl agree on the lock-file paths.

⚡ Performance: ✅ LGTM

There is no runtime performance impact: the change only moves files and updates paths.


📝 Documentation: ✅ LGTM

The path references in the documentation match the new layout.


Reviewed commit: 69ae8766

@ryanking13
ryanking13 merged commit d89edf6 into main Nov 13, 2026
23 checks passed
@ryanking13
ryanking13 deleted the gyeongjae/pyodide-bundle-dir branch November 13, 2026 05:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants