compiletest: show output in debug logging

[jyn514]

I had a test I was confused by; the root issue is that error-pattern runs before normalization, even though //~ ERROR runs after normalization. This logging caught the issue immediately.

[rustbot]

Some changes occurred in src/tools/compiletest

cc @jieyouxu

[rustbot]

r? @Zalathar

rustbot has assigned @Zalathar.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[hkBst]

This change simply adds output_to_check onto an existing debug log.

@jyn514 can you show an example of the change in output that results from this?

View changes since this review

[Zalathar]

I’m vaguely worried about this being too verbose; how does the output look in practice? Maybe it’s actually fine?

[jyn514]

the output varies wildly depending on the test. here's an example for a small test with //~ ERROR annotations:

us(256)) props.error_patterns=[] output_to_check="{\"$message_type\":\"diagnostic\",\"message\":\"failed to resolve: could not find `bar` in the list of imported crates\",\"code\":{\"code\":\"E0433\",\"explanation\":\"An undeclared crate, module, or type was used.\\n\\nErroneous code example:\\n\\n```compile_fail,E0433\\nlet map = HashMap::new();\\n// error: failed to resolve: use of undeclared type `HashMap`\\n```\\n\\nPlease verify you didn't misspell the type/module's name or that you didn't\\nforget to import it:\\n\\n```\\nuse std::collections::HashMap; // HashMap has been imported.\\nlet map: HashMap<u32, u32> = HashMap::new(); // So it can be used!\\n```\\n\\nIf you've expected to use a crate name:\\n\\n```compile_fail\\nuse ferris_wheel::BigO;\\n// error: failed to resolve: use of undeclared module or unlinked crate\\n```\\n\\nMake sure the crate has been added as a dependency in `Cargo.toml`.\\n\\nTo use a module from your current crate, add the `crate::` prefix to the path.\\n\"},\"level\":\"error\",\"spans\":[{\"file_name\":\"/Users/jyn/src/ferrocene2/tests/ui/resolve/visibility-indeterminate.rs\",\"byte_start\":84,\"byte_end\":87,\"line_start\":5,\"line_end\":5,\"column_start\":10,\"column_end\":13,\"is_primary\":true,\"text\":[{\"text\":\"pub(in ::bar) struct Baz {} //~ ERROR failed to resolve: could not find `bar` in the list of imported crates\",\"highlight_start\":10,\"highlight_end\":13}],\"label\":\"could not find `bar` in the list of imported crates\",\"suggested_replacement\":null,\"suggestion_applicability\":null,\"expansion\":null}],\"children\":[],\"rendered\":\"error[E0433]: failed to resolve: could not find `bar` in the list of imported crates\\n  --> /Users/jyn/src/ferrocene2/tests/ui/resolve/visibility-indeterminate.rs:5:10\\n   |\\nLL | pub(in ::bar) struct Baz {} //~ ERROR failed to resolve: could not find `bar` in the list of imported crates\\n   |          ^^^ could not find `bar` in the list of imported crates\\n\\n\"}\n{\"$message_type\":\"diagnostic\",\"message\":\"cannot find macro `foo` in this scope\",\"code\":null,\"level\":\"error\",\"spans\":[{\"file_name\":\"/Users/jyn/src/ferrocene2/tests/ui/resolve/visibility-indeterminate.rs\",\"byte_start\":18,\"byte_end\":21,\"line_start\":3,\"line_end\":3,\"column_start\":1,\"column_end\":4,\"is_primary\":true,\"text\":[{\"text\":\"foo!(); //~ ERROR cannot find macro `foo` in this scope\",\"highlight_start\":1,\"highlight_end\":4}],\"label\":null,\"suggested_replacement\":null,\"suggestion_applicability\":null,\"expansion\":null}],\"children\":[],\"rendered\":\"error: cannot find macro `foo` in this scope\\n  --> /Users/jyn/src/ferrocene2/tests/ui/resolve/visibility-indeterminate.rs:3:1\\n   |\\nLL | foo!(); //~ ERROR cannot find macro `foo` in this scope\\n   | ^^^\\n\\n\"}\n{\"$message_type\":\"diagnostic\",\"message\":\"aborting due to 2 previous errors\",\"code\":null,\"level\":\"error\",\"spans\":[],\"children\":[],\"rendered\":\"error: aborting due to 2 previous errors\\n\\n\"}\n{\"$message_type\":\"diagnostic\",\"message\":\"For more information about this error, try `rustc --explain E0433`.\",\"code\":null,\"level\":\"failure-note\",\"spans\":[],\"children\":[],\"rendered\":\"For more information about this error, try `rustc --explain E0433`.\\n\"}\n"

The original use case I had was for an error-pattern test, where it's much smaller:

2025-10-27T23:51:10.982056Z DEBUG compiletest::runtest::ui: run_ui_test: explicit=false config.compare_mode=None proc_res.status=ExitStatus(unix_wait_stat
us(0)) props.error_patterns=["explicit panic"] output_to_check="\nthread 'main' (39068430) panicked at /Users/jyn/src/ferrocene2/tests/ui/fn/expr-fn-panic.rs:6:5:\nexplicit panic\nnote: run with `RUST_BACKTRACE=1` environment variable to display a backtrace\n"

Perhaps this could check if it's parsing JSON and only show the message if so?

---

I do want to note that compiletest is already extremely verbose when passed RUST_LOG=debug:

RUST_LOG=debug x t tests/ui/fn/expr-fn-panic.rs --force-rerun 2>&1 | wc -l
   36456

so I don't feel like this is that much of a drawback ...

[Zalathar]

Sounds good enough for me. This is a low-stakes change, so let’s just merge it; if someone finds it annoying later they can propose limiting or removing it at that time.

@bors r+ rollup

[bors]

📌 Commit 5976015 has been approved by Zalathar

It is now in the queue for this repository.