various fixes for scalable vectors#153286
Open
davidtwco wants to merge 7 commits intorust-lang:mainfrom
Open
Conversation
Collaborator
Collaborator
|
|
This comment has been minimized.
This comment has been minimized.
ed74abe to
dc9af49
Compare
Collaborator
|
This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed. Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers. |
This comment has been minimized.
This comment has been minimized.
RalfJung
reviewed
Mar 2, 2026
RalfJung
reviewed
Mar 2, 2026
RalfJung
reviewed
Mar 2, 2026
RalfJung
reviewed
Mar 2, 2026
Amanieu
reviewed
Mar 2, 2026
dc9af49 to
06b46c4
Compare
This comment was marked as resolved.
This comment was marked as resolved.
06b46c4 to
4329945
Compare
Collaborator
|
cc @wesleywiser |
This comment was marked as resolved.
This comment was marked as resolved.
Introduces a new check in `rustc_hir_analysis` enforcing that tuples of scalable vectors should only have two, three or four vectors.
4329945 to
ab6b50b
Compare
Collaborator
Renaming to remove any ambiguity as to what "vector" refers to in this context
Renaming to remove any ambiguity as to what "vector" refers to in this context.
ab6b50b to
5fc0060
Compare
This comment was marked as resolved.
This comment was marked as resolved.
5fc0060 to
f8b4bc8
Compare
This comment has been minimized.
This comment has been minimized.
Instead of just using regular struct lowering for these types, which results in an incorrect ABI (e.g. returning indirectly), use `BackendRepr::ScalableVector` which will lower to the correct type and be passed in registers. This also enables some simplifications for generating alloca of scalable vectors and greater re-use of `scalable_vector_parts`. A LLVM codegen test demonstrating the changed IR this generates is included in the next commit alongside some intrinsics that make these tuples usable.
Clang changed from representing tuples of scalable vectors as
structs rather than as wide vectors (that is, scalable vector types
where the `N` part of the `<vscale x N x ty>` type was multiplied by
the number of vectors). rustc mirrored this in the initial implementation
of scalable vectors.
Earlier versions of our patches used the wide vector representation and
our intrinsic patches used the legacy
`llvm.aarch64.sve.tuple.{create,get,set}{2,3,4}` intrinsics for creating
these tuples/getting/setting the vectors, which were only supported
due to LLVM's `AutoUpgrade` pass converting these intrinsics into
`llvm.vector.insert`. `AutoUpgrade` only supports these legacy intrinsics
with the wide vector representation.
With the current struct representation, Clang has special handling in
codegen for generating `insertvalue`/`extractvalue` instructions for
these operations, which must be replicated by rustc's codegen for our
intrinsics to use. This patch implements new intrinsics in
`core::intrinsics::scalable` (mirroring the structure of
`core::intrinsics::simd`) which rustc lowers to the appropriate
`insertvalue`/`extractvalue` instructions.
cfb2032 to
8ba075f
Compare
This comment was marked as resolved.
This comment was marked as resolved.
8ba075f to
6b19e1a
Compare
This comment was marked as resolved.
This comment was marked as resolved.
Generate debuginfo for scalable vectors, following the structure that Clang generates for scalable vectors.
6b19e1a to
5b19574
Compare
This comment has been minimized.
This comment has been minimized.
Abstract over the existing `simd_cast` intrinsic to implement a new `sve_cast` intrinsic - this is better than allowing scalable vectors to be used with all of the generic `simd_*` intrinsics.
5b19574 to
39d3880
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
These are a handful of patches fixing bugs with the current
#[rustc_scalable_vector]infrastructure so that we can upstream the intrinsics to stdarch:Introduces a new check in
rustc_hir_analysisenforcing that tuples of scalable vectors should only have two, three or four vectors.Instead of just using regular struct lowering for tuples of scalable vectors, which results in an incorrect ABI (e.g. returning indirectly), we change to using
BackendRepr::ScalableVectorwhich will lower to the correct type and be passed in registers.Clang changed from representing tuples of scalable vectors as structs rather than as wide vectors (that is, scalable vector types where the
Npart of the<vscale x N x ty>type was multiplied by the number of vectors). rustc mirrored this in the initial implementation of scalable vectors that didn't land.When our early patches used the wide vector representation, our intrinsic patches used the legacy
llvm.aarch64.sve.tuple.{create,get,set}{2,3,4}intrinsics for creating these tuples/getting/setting the vectors, which were only supported due to LLVM'sAutoUpgradepass converting these intrinsics intollvm.vector.insert.AutoUpgradeonly supports these legacy intrinsics with the wide vector representation.With the current struct representation, Clang has special handling in codegen for generating
insertvalue/extractvalueinstructions for these operations, which must be replicated by rustc's codegen for our intrinsics to use.We add a new
core::intrinsics::scalablemodule (never intended to be stable, just used by the stdarch intrinsics, gated bycore_intrinsics) and add new intrinsics which rustc lowers to the appropriateinsertvalue/extractvalueinstructions.Add generation of debuginfo for scalable vectors, following the DWARF that Clang generates.
Some intrinsics need something like
simd_cast, which will work for scalable vectors too, so this implements a newsve_castintrinsic that uses the same internals assimd_cast. This seemed better than permitting scalable vectors to be used with thesimd_*intrinsics more generally as I can't guarantee this would work for all of them.This is a relatively large patch but most of it is tests, and each commit should be relatively standalone. It's a little bit easier to upstream them together to avoid needing to stack them. It's possible that some more compiler fixes will be forthcoming but it's looking like this might be all at the moment.
r? @workingjubilee (discussed on Zulip)