Distribute cg_clif as rustup component on the nightly channel by bjorn3 · Pull Request #81746 · rust-lang/rust

bjorn3 · 2021-02-04T10:26:21Z

This makes it possible to use cg_clif using:

$ rustup component add rustc-codegen-cranelift-preview --toolchain nightly
$ RUSTFLAGS="-Zcodegen-backend=cranelift" cargo +nightly build

cc rust-lang/compiler-team#405.
r? @Mark-Simulacrum

bjorn3 · 2021-02-08T08:33:11Z

./x.py build seems to build another rustc stage now. Why does that happen?

src/bootstrap/dist.rs

Mark-Simulacrum · 2021-02-12T02:41:59Z

For the bootstrap tests, I'd recommend an x.py build locally that tries to reproduce it (--dry-run should help with getting quick results), and running with -vvv to see what is invoking the newly added rustc builds. I suspect it's something in the codegenbackend step in compile.rs, but haven't investigated myself (and would prefer not to).

bjorn3 · 2021-02-12T11:00:47Z

--dry-run should help with getting quick results

Thanks for the tip!

I got this with download-ci-llvm:

$ ./x.py build --dry-run -v
[...]
running: /home/bjorn/Documenten/rust/build/bootstrap/debug/bootstrap build --dry-run -v
thread 'main' panicked at 'std::fs::read_to_string(ci_llvm.join("link-type.txt")) failed with No such file or directory (os error 2)', src/bootstrap/config.rs:823:33
stack backtrace:
   0: rust_begin_unwind
             at /rustc/05b6023675d77979637b04a350c85903fbf59257/library/std/src/panicking.rs:493:5
   1: std::panicking::begin_panic_fmt
             at /rustc/05b6023675d77979637b04a350c85903fbf59257/library/std/src/panicking.rs:435:5
   2: bootstrap::config::Config::parse
             at ./src/bootstrap/config.rs:823:33
   3: bootstrap::main
             at ./src/bootstrap/bin/main.rs:14:18

Disabling it fixes the problem.

bjorn3 · 2021-02-12T13:05:44Z

Previously it would build assemble stage 1 -> (rustc stage 0 and codegen backend stage 0) and std stage 1. Now it also builds codegen backend stage 1 -> rustc stage 1. Removing the DEFAULT override in the Step implementation fixed it.

bjorn3 · 2021-02-12T13:51:01Z

Let's check if the rustup component is correctly generated.

@bors try

bors · 2021-02-12T13:51:17Z

⌛ Trying commit 909dbc1857b49279156b64061c13ab55c9f03b80 with merge 4bbf0fa74cbc11268c4f6f9d641e12b360065fc1...

bors · 2021-02-12T14:40:23Z

☀️ Try build successful - checks-actions
Build commit: 4bbf0fa74cbc11268c4f6f9d641e12b360065fc1 (4bbf0fa74cbc11268c4f6f9d641e12b360065fc1)

bjorn3 · 2021-02-12T15:09:29Z

I must have made a mistake somewhere. rustup-toolchain-install-master 4bbf0fa74cbc11268c4f6f9d641e12b360065fc1 -c rustc-codegen-cranelift fails, while -c miri and -c rust-analyzer work.

bjorn3 · 2021-02-13T18:37:20Z

I think I should wire up compiler/rustc_codegen_cranelift to the tidy license checking. It isn't currently checked due to being in a different workspace as far as I know.

bjorn3 · 2021-02-19T17:07:47Z

@bors try

bors · 2021-02-19T17:07:58Z

⌛ Trying commit de705b10023b1ed5b9ad92810e3a0ea1dc29caee with merge 874d9737dc9cca284c21e5b4aaf40bf57502947f...

bors · 2023-10-27T19:02:26Z

⌛ Trying commit 0ab5ba1 with merge 4d3f1db...

bors · 2023-10-27T19:40:18Z

☀️ Try build successful - checks-actions
Build commit: 4d3f1db (4d3f1db3d62e811f9bfb08ff864f86e6d804a365)

bjorn3 · 2023-10-27T19:42:45Z

And to try if I didn't break anything on linux:

@bors try

bors · 2023-10-27T19:43:53Z

⌛ Trying commit aaa4e54 with merge 4629770...

bors · 2023-10-27T21:06:22Z

☀️ Try build successful - checks-actions
Build commit: 4629770 (46297703790ab134bb33494856710b67f29471ff)

bjorn3 · 2023-10-28T08:03:38Z

@Mark-Simulacrum could I get another review? Only the last commit is new compared to the last attempt at merging.

Mark-Simulacrum · 2023-10-28T14:14:30Z

@bors r+

bors · 2023-10-28T14:14:32Z

📌 Commit aaa4e54 has been approved by Mark-Simulacrum

It is now in the queue for this repository.

bors · 2023-10-28T15:16:30Z

⌛ Testing commit aaa4e54 with merge 6a66ca2...

bors · 2023-10-28T17:10:35Z

☀️ Test successful - checks-actions
Approved by: Mark-Simulacrum
Pushing 6a66ca2 to master...

lqd · 2023-10-28T17:11:06Z

This PR was opened 2½ years ago, on Feb 4 2021. Huge congratulations @bjorn3, for this amazing achievement (and it's only the beginning).

jhpratt · 2023-10-28T17:12:19Z

Outstanding work! I look forward to using this. Be proud 🥳

bjorn3 · 2023-10-28T18:12:25Z

I found two issues:

The CODEGEN_BACKENDS env var doesn't get passed from the CI config to src/ci/run.sh, so we don't actually ship cg_clif.
For some reason despite the above we still create a rustc-codegen-cranelift-preview rustup component, but it is missing the codegen backend.

Currently trying to find the cause of both issues.

Edit: #117328 seems to be a fix for both issues.

rust-timer · 2023-10-28T18:33:37Z

Finished benchmarking commit (6a66ca2): comparison URL.

Overall result: ❌ regressions - no action needed

@rustbot label: -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.2%	[0.2%, 0.2%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Max RSS (memory usage)

This benchmark run did not return any relevant results for this metric.

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 636.535s -> 637.097s (0.09%)
Artifact size: 304.40 MiB -> 304.41 MiB (0.00%)

tgross35 · 2023-10-30T17:58:05Z

I can't be the only one with this question - how does one actually put this to use after rustup component add rustc-codegen-cranelift-x86_64-unknown-linux-gnu?

Edit: I thought #117328 landed by rustc 1.75.0-nightly (608e9682f 2023-10-29) but I guess not. Once it does, seems it will just work by passing rustc -Zcodegen-backend=??

Skgland · 2023-10-30T19:00:21Z

I can't be the only one with this question - how does one actually put this to use after rustup component add rustc-codegen-cranelift-x86_64-unknown-linux-gnu?

That looks like you are mixing the cranelift codegen backend component with the compilation target toolchain.

Edit: I thought #117328 landed by rustc 1.75.0-nightly (608e9682f 2023-10-29) but I guess not. Once it does, seems it will just work by passing rustc -Zcodegen-backend=??

As github sais this was merged two days ago I would have expected this to be in the current nightly and based on the PR description I would expect the following to work

depending on whether nightly is installed
a. rustup toolchain install nightly
b. rustup toolchain update
rustup component add rustc-codegen-cranelift-preview --toolchain nightly
RUSTFLAGS="-Zcodegen-backend=cranelift" cargo +nightly build

1b and 2 succeeded the later suprisingly quickly, but 3 failed with

$ RUSTFLAGS="-Zcodegen-backend=cranelift" cargo +nightly build
error: failed to run `rustc` to learn about target-specific information

Caused by:
  process didn't exit successfully: `/home/bennet/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/rustc - --crate-name ___ --print=file-names -Zcodegen-backend=cranelift --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=split-debuginfo --print=crate-name --print=cfg` (exit status: 1)
  --- stderr
  error: failed to find a `codegen-backends` folder in the sysroot candidates:
         * /home/bennet/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu
         * /home/bennet/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu

Kobzol · 2023-10-30T19:02:42Z

The build of the cranelift component wasn't working in this PR. It was fixed in #117328, so it should hopefully work in tomorrow's nightly.

tgross35 · 2023-10-30T19:04:49Z

I can't be the only one with this question - how does one actually put this to use after rustup component add rustc-codegen-cranelift-x86_64-unknown-linux-gnu?

That looks like you are mixing the cranelift codegen backend component with the compilation target toolchain.

I think rustc-codegen-cranelift-preview is just a shortcut to the component rustc-codegen-cranelift-x86_64-unknown-linux-gnu (host target as applicable), I just didn't know about it since it doesn't show up in rustup component list

Edit: I thought #117328 landed by rustc 1.75.0-nightly (608e9682f 2023-10-29) but I guess not. Once it does, seems it will just work by passing rustc -Zcodegen-backend=??

As github sais this was merged two days ago I would have expected this to be in the current nightly and based on the PR description I would expect the following to work

Opened two days ago but merged 15 hours ago

g2p · 2023-10-30T21:30:23Z

~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/manifest-rustc-codegen-cranelift-preview-x86_64-unknown-linux-gnu is just a handful of documentation files, until the pipeline is entirely working.

tgross35 · 2023-10-31T17:16:07Z

2023-10-30 nightly indeed works! rustc -Zcodegen-backend=cranelift / RUSTFLAGS=-Zcodegen-backend=cranelift cargo build

lqd · 2023-10-31T17:23:29Z

@tgross35 https://bjorn3.github.io/2023/10/31/progress-report-oct-2023.html also describes the couple of other ways you can use cargo's support for selecting the backend to use.

bjorn3 added T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. A-cranelift Things relevant to the [future] cranelift backend labels Feb 4, 2021

rust-highfive assigned Mark-Simulacrum Feb 4, 2021

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Feb 4, 2021

This comment has been minimized.

Sign in to view

bjorn3 force-pushed the cg_clif_rustup_component branch from 077c3ee to b64122f Compare February 4, 2021 10:54

This comment has been minimized.

Sign in to view

bjorn3 force-pushed the cg_clif_rustup_component branch from b64122f to 06d3150 Compare February 4, 2021 11:12

This comment has been minimized.

Sign in to view

bjorn3 force-pushed the cg_clif_rustup_component branch from 8b5ccb9 to c341ecf Compare February 8, 2021 08:54

Mark-Simulacrum reviewed Feb 12, 2021

View reviewed changes

src/bootstrap/dist.rs Outdated Show resolved Hide resolved

Mark-Simulacrum added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 12, 2021

bjorn3 force-pushed the cg_clif_rustup_component branch from c341ecf to cd173e6 Compare February 12, 2021 13:05

bjorn3 mentioned this pull request Feb 13, 2021

Add a profile option to select the codegen backend rust-lang/cargo#9118

Merged

bors mentioned this pull request Oct 28, 2023

Run part of rustc_codegen_gcc's tests in CI #117313

Merged

lqd mentioned this pull request Oct 28, 2023

pass CODEGEN_BACKENDS to docker #117328

Merged

kayabaNerve mentioned this pull request Oct 29, 2023

Review using Cranelift in CI serai-dex/serai#410

Open

oxalica mentioned this pull request Oct 30, 2023

Add rustc-codegen-cranelift component to rust-overlay for nightly oxalica/rust-overlay#142

Closed

github-actions bot mentioned this pull request Oct 31, 2023

Automated pull from upstream master ferrocene/ferrocene#74

Merged

lqd mentioned this pull request Oct 31, 2023

CI: Bootstrap no longer documents compiler crates #117430

Closed

Kobzol mentioned this pull request Nov 9, 2023

Add Rustup installation instructions to README rust-lang/rustc_codegen_cranelift#1421

Merged

rami3l mentioned this pull request Apr 5, 2024

Distribute GCC backend rust-lang/rustup#3756

Open

bjorn3 mentioned this pull request Aug 16, 2024

Distribute rustc_codegen_cranelift for Windows #128939

Merged

Uh oh!

Conversation

bjorn3 commented Feb 4, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

bjorn3 commented Feb 8, 2021

Uh oh!

This comment has been minimized.

Uh oh!

Mark-Simulacrum commented Feb 12, 2021

Uh oh!

bjorn3 commented Feb 12, 2021

Uh oh!

bjorn3 commented Feb 12, 2021

Uh oh!

bjorn3 commented Feb 12, 2021

Uh oh!

bors commented Feb 12, 2021

Uh oh!

bors commented Feb 12, 2021

Uh oh!

bjorn3 commented Feb 12, 2021

Uh oh!

bjorn3 commented Feb 13, 2021

Uh oh!

bjorn3 commented Feb 19, 2021

Uh oh!

bors commented Feb 19, 2021

Uh oh!

bors commented Oct 27, 2023

Uh oh!

bors commented Oct 27, 2023

Uh oh!

bjorn3 commented Oct 27, 2023

Uh oh!

bors commented Oct 27, 2023

Uh oh!

bors commented Oct 27, 2023

Uh oh!

bjorn3 commented Oct 28, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Mark-Simulacrum commented Oct 28, 2023

Uh oh!

bors commented Oct 28, 2023

Uh oh!

bors commented Oct 28, 2023

Uh oh!

bors commented Oct 28, 2023

Uh oh!

lqd commented Oct 28, 2023

Uh oh!

jhpratt commented Oct 28, 2023

Uh oh!

bjorn3 commented Oct 28, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rust-timer commented Oct 28, 2023

Overall result: ❌ regressions - no action needed

Instruction count

Max RSS (memory usage)

Cycles

Binary size

Uh oh!

tgross35 commented Oct 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Skgland commented Oct 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Kobzol commented Oct 30, 2023

Uh oh!

tgross35 commented Oct 30, 2023

Uh oh!

g2p commented Oct 30, 2023

Uh oh!

bjorn3 commented Feb 4, 2021 •

edited

Loading

bjorn3 commented Oct 28, 2023 •

edited

Loading

bjorn3 commented Oct 28, 2023 •

edited

Loading

tgross35 commented Oct 30, 2023 •

edited

Loading

Skgland commented Oct 30, 2023 •

edited

Loading