Fix variable deallocation order in panic unwinding paths

[sladyn98]

View all comments

This PR fixes a soundness bug where local variables are deallocated out of order during panic unwinding, allowing destructors to access freed memory. This violates Rust's safety guarantees and has caused real-world unsoundness in crates like generatively.

This PR removes the is_generator check and unconditionally emits StorageDead statements during unwinding for ALL functions, bringing non-generator behavior in line with generators. It ensures that during unwinding, when a local variable goes out of scope, its storage is properly marked as dead via StorageDead, allowing the borrow checker to enforce the
invariant that values must outlive their references even in panic paths.

Fixes #147875

[rustbot]

r? @wesleywiser

rustbot has assigned @wesleywiser.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[sladyn98]

r? @dianne

[dianne]

It looks like there's a bug causing an assertion failure when building the standard library. I've given it a look and offered a guess at what's causing it below. There's still work to do here beyond fixing that, though.

First, could you add a ui test to demonstrate that this fixes #147875? It looks like it might not yet, since the code for scheduling unwind drops on calls panicking looks unchanged.

Second, after verifying that this results in the correct borrow-checking behavior, we need to make sure that this change doesn't negatively affect codegen. Per the old comment on needs_cleanup, at least at the time it was written, LLVM didn't handle the unnecessary cleanup blocks and StorageDeads particularly well. If you can demonstrate with codegen tests that that's not an issue anymore, and perf isn't too bad, that might be all that's needed. But my expectation is that we'll have to get rid of or ignore the StorageDeads later in compilation (sometime after they serve their purpose in borrowck). Unless there's a reason to keep the StorageDeads around longer, my gut feeling is that this cleanup would be best as a post-borrowck MIR pass (maybe as part of CleanupPostBorrowck?), since then optimization passes can be done on cleaner MIR and we can test it works with MIR tests rather than codegen tests. Could you also add a test for this not affecting later stages of compilation? If you accomplish that by removing the unwind-path StorageDeads as part of a MIR pass, that'd be a mir-opt test.

Before you push again, you'll probably want to run the codegen and mir-opt tests to make sure the former is clean and to bless the latter. Regardless of what approach we take here, if we're changing how the MIR is built, there should be differences in the MIR building test output (part of the mir-opt suite).

View changes since this review

[rustbot]

Reminder, once the PR becomes ready for a review, use @rustbot ready.

[dianne]

Also, could you change the PR description? #147875 on its own doesn't allow destructors to access freed memory, it doesn't allow for the creation of dangling references, and I'm at least not aware of a safety guarantee that it violates. You should only get unsoundness out of it if you write unsafe code on the assumption that the borrow checker will enforce the relative drop order of locals that may have destructors and those that definitely don't. Of course, per language team decision, consistent drop order is a promise Rust would like to make. But it's not quite the same as the borrow-checker failing to ensure places outlive their references.

[sladyn98]

So what i did was write this simple rust program panic drop.rs

 //@ compile-flags: -C no-prepopulate-passes

#![crate_type = "lib"]

#[no_mangle]
pub fn function_with_drops() {

    let _a = String::from("first");
    let _b = String::from("second");
    let _c = String::from("third");
    might_panic();
}

#[inline(never)]
pub fn might_panic() {
    // This might panic
    panic!()
}

I ran the llvm to get the intermediate representaion and on looking at the IR I cannot find any llvm.lifetime.end statements suggesting to us that on master the StorageDead statements are missing, which according to my understanding means that the borrowchecker does not know when the storage becomes invalid. Let me now write the UI test to see what is up

[dianne]

StorageDead is indeed not present there on the main branch, but that doesn't create a soundness hole in the borrow-checker; those locals are simply treated as being maybe live until the end of the stack frame, so it's technically sound for destructors on the unwind path to reference them. When unwinding, the stack frame will be popped anyway, so we don't need to hint to llvm that the memory's no longer in use. The issue is that we'd like to be more strict in the borrow-checker, both for consistency and so that unsafe code can rely on drop order being enforced.

edit: adjusted wording

[dianne]

This still needs CI to pass before I can review it properly. I've left a few comments on obvious things, but I don't think reviewing the code changes would be helpful at this point. Please test your changes locally. You don't have to run the whole test suite yourself, but for this change, you'll at least want make sure that the mir-opt and codegen tests all pass, that any relevant ui tests pass, and that tidy passes as well.

Could you rebase onto a more recent commit, also? I don't expect there will be conflicts in the MIR building part of this, but I'm not sure about the rest.

I don't mean to be harsh, but this is a relatively complex and nuanced change. If you're not familiar with what's being changed, why it's being changed, the consequences/needs of that, and general contribution procedure, I'd recommend gaining familiarity with easier issues instead.

View changes since this review

[rustbot]

Some changes occurred to MIR optimizations

cc @rust-lang/wg-mir-opt

[rustbot]

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

[rust-log-analyzer]

The job x86_64-gnu-gcc failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)

/dev/sda15      105M  6.2M   99M   6% /boot/efi
tmpfs           1.6G   12K  1.6G   1% /run/user/1001
================================================================================

Sufficient disk space available (94917288KB >= 52428800KB). Skipping cleanup.
##[group]Run echo "[CI_PR_NUMBER=$num]"
echo "[CI_PR_NUMBER=$num]"
shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
---
[RUSTC-TIMING] git2 test:false 16.723
error[E0597]: `counter` does not live long enough
   --> src/tools/cargo/src/cargo/sources/git/utils.rs:910:25
    |
864 |             let mut counter = MetricsCounter::<10>::new(0, last_update);
    |                 ----------- binding `counter` declared here
...
886 |             rcb.transfer_progress(|stats| {
    |                                   ------- value captured here
...
910 |                         counter.add(stats.received_bytes(), now);
    |                         ^^^^^^^ borrowed value does not live long enough
...
926 |         });
    |         -
    |         |
    |         `counter` dropped here while still borrowed
    |         borrow might be used here, when `rcb` is dropped and runs the destructor for type `RemoteCallbacks<'_>`
    |
    = note: values in a scope are dropped in the opposite order they are defined

For more information about this error, try `rustc --explain E0597`.
[RUSTC-TIMING] cargo test:false 15.604

For more information how to resolve CI failures of this job, visit this link.

[dianne]

Well, that's awkward. I took another look at the r-a error too, and I think it and the cargo error are the same thing: a Drop local is modified to reference a later-declared non-Drop local, then moved so it's dropped before the non-Drop local is deallocated. Previously this worked fine. After this change, if there's a function call before the move happens, the non-Drop local is StorageDead on the call's unwind path before the Drop local's destructor is called, even though that's not the order on the happy path. Given that this has come up multiple times in PR CI, I imagine it'd be fairly common in the wild.

I've only looked at the smaller test diffs so far, but I have some comments I'd like addressed before I dig deeper.

@rustbot author

View changes since this review

[dianne]

Why's there an extra StorageDead on the normal execution path here?

[dianne]

What is this testing? A test for the change in static semantics should be something that compiled before this change and doesn't compile after, like the example in #147875 or a minimization of the r-a or cargo failures in CI.

[dianne]

I imagine this is from an old revision?

[dianne]

Do you know what's going on here?