Skip to content

Conversation

Zalathar
Copy link
Contributor

@Zalathar Zalathar commented Sep 2, 2025

Thanks to the efforts on #140192, compiletest no longer has an unstable dependency on libtest, but it still has an unstable dependency on #![feature(internal_output_capture)]. That makes building compiletest more complicated than for most other bootstrap tools.

This PR therefore adds opt-in support for an experimental compiletest mode that avoids the use of internal_output_capture APIs, and instead uses more mundane means to capture the output of individual test runners.

Each TestCx now contains &dyn ConsoleOut references for stdout and stderr. All print statements in compiletests::runtest have been replaced with write! or writeln! calls that explicitly write to one of those trait objects. The underlying implementation then forwards to print! or eprint! (for --no-capture or old-output-capture mode), or writes to a separate buffer (in new-output-capture mode).


Currently, new-output-capture is disabled by default. It can be explicitly enabled in one of two ways:

  • When running x test, pass --new-output-capture=on as a compiletest argument (after --).
    • E.g. x test ui -- --new-output-capture=on.
    • The short form is -Non or -Ny.
  • Set environment variable COMPILETEST_NEW_OUTPUT_CAPTURE=on.

After some amount of opt-in testing, new-output-capture will become the default (with a temporary opt-out). Eventually, old-output-capture and #![feature(internal_output_capture)] will be completely removed from compiletest.

r? jieyouxu

@rustbot rustbot added A-compiletest Area: The compiletest test runner A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Sep 2, 2025
@rustbot
Copy link
Collaborator

rustbot commented Sep 2, 2025

Some changes occurred in src/tools/compiletest

cc @jieyouxu

This PR modifies bootstrap.example.toml.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

@Zalathar

This comment was marked as outdated.

@Kobzol
Copy link
Member

Kobzol commented Sep 2, 2025

I think that if this is an experimental feature designed for testing with a handful of people, I wouldn't add it to bootstrap.toml (nor add a change tracker entry), and just use some magic environment variable for it. bootstrap.toml should ideally be used for configuration that will last long-term, as it's rather annoying to remove stuff from it.

@Zalathar
Copy link
Contributor Author

Zalathar commented Sep 2, 2025

Fair enough; I'll switch over to an environment variable instead.

Copy link
Member

@jieyouxu jieyouxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this! Other changes look good to me, just a tiny nit regarding "please don't support so many ways to spell a true/false" of the flag lol

View changes since this review

@jieyouxu
Copy link
Member

jieyouxu commented Sep 3, 2025

@rustbot author

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 3, 2025
@rustbot
Copy link
Collaborator

rustbot commented Sep 3, 2025

Reminder, once the PR becomes ready for a review, use @rustbot ready.

@jieyouxu
Copy link
Member

jieyouxu commented Sep 4, 2025

@bors r+ rollup

@bors
Copy link
Collaborator

bors commented Sep 4, 2025

📌 Commit 9c5e1cb has been approved by jieyouxu

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Sep 4, 2025
@Zalathar
Copy link
Contributor Author

Zalathar commented Sep 4, 2025

Added a short comment to parse_bool_option, since the topic of which values to accept came up in review.

@bors r=jieyouxu

@bors
Copy link
Collaborator

bors commented Sep 4, 2025

📌 Commit c2ae2e0 has been approved by jieyouxu

It is now in the queue for this repository.

Zalathar added a commit to Zalathar/rust that referenced this pull request Sep 4, 2025
compiletest: Implement an experimental `--new-output-capture` mode

Thanks to the efforts on rust-lang#140192, compiletest no longer has an unstable dependency on libtest, but it still has an unstable dependency on `#![feature(internal_output_capture)]`. That makes building compiletest more complicated than for most other bootstrap tools.

This PR therefore adds opt-in support for an experimental compiletest mode that avoids the use of `internal_output_capture` APIs, and instead uses more mundane means to capture the output of individual test runners.

Each `TestCx` now contains `&dyn ConsoleOut` references for stdout and stderr. All print statements in `compiletests::runtest` have been replaced with `write!` or `writeln!` calls that explicitly write to one of those trait objects. The underlying implementation then forwards to `print!` or `eprint!` (for `--no-capture` or old-output-capture mode), or writes to a separate buffer (in new-output-capture mode).

---

Currently, new-output-capture is disabled by default. It can be explicitly enabled in one of two ways:

- When running `x test`, pass `--new-output-capture=on` as a *compiletest* argument (after `--`).
  - E.g. `x test ui -- --new-output-capture=on`.
  - The short form is `-Non` or `-Ny`.
- Set environment variable `COMPILETEST_NEW_OUTPUT_CAPTURE=on`.

After some amount of opt-in testing, new-output-capture will become the default (with a temporary opt-out). Eventually, old-output-capture and `#![feature(internal_output_capture)]` will be completely removed from compiletest.

r? jieyouxu
bors added a commit that referenced this pull request Sep 4, 2025
Rollup of 5 pull requests

Successful merges:

 - #145682 (Promote aarch64-pc-windows-msvc to Tier 1)
 - #145690 (Implement Integer funnel shifts)
 - #146119 (compiletest: Implement an experimental `--new-output-capture` mode)
 - #146168 (Update bootstrap's dependencies to remove winapi and old windows-sys)
 - #146182 (Don't require next-solver `ProbeRef` to be `Copy`)

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit be1d829 into rust-lang:master Sep 4, 2025
10 checks passed
@rustbot rustbot added this to the 1.91.0 milestone Sep 4, 2025
rust-timer added a commit that referenced this pull request Sep 4, 2025
Rollup merge of #146119 - Zalathar:capture, r=jieyouxu

compiletest: Implement an experimental `--new-output-capture` mode

Thanks to the efforts on #140192, compiletest no longer has an unstable dependency on libtest, but it still has an unstable dependency on `#![feature(internal_output_capture)]`. That makes building compiletest more complicated than for most other bootstrap tools.

This PR therefore adds opt-in support for an experimental compiletest mode that avoids the use of `internal_output_capture` APIs, and instead uses more mundane means to capture the output of individual test runners.

Each `TestCx` now contains `&dyn ConsoleOut` references for stdout and stderr. All print statements in `compiletests::runtest` have been replaced with `write!` or `writeln!` calls that explicitly write to one of those trait objects. The underlying implementation then forwards to `print!` or `eprint!` (for `--no-capture` or old-output-capture mode), or writes to a separate buffer (in new-output-capture mode).

---

Currently, new-output-capture is disabled by default. It can be explicitly enabled in one of two ways:

- When running `x test`, pass `--new-output-capture=on` as a *compiletest* argument (after `--`).
  - E.g. `x test ui -- --new-output-capture=on`.
  - The short form is `-Non` or `-Ny`.
- Set environment variable `COMPILETEST_NEW_OUTPUT_CAPTURE=on`.

After some amount of opt-in testing, new-output-capture will become the default (with a temporary opt-out). Eventually, old-output-capture and `#![feature(internal_output_capture)]` will be completely removed from compiletest.

r? jieyouxu
@Zalathar Zalathar deleted the capture branch September 4, 2025 13:11
github-actions bot pushed a commit to rust-lang/miri that referenced this pull request Sep 5, 2025
Rollup of 5 pull requests

Successful merges:

 - rust-lang/rust#145682 (Promote aarch64-pc-windows-msvc to Tier 1)
 - rust-lang/rust#145690 (Implement Integer funnel shifts)
 - rust-lang/rust#146119 (compiletest: Implement an experimental `--new-output-capture` mode)
 - rust-lang/rust#146168 (Update bootstrap's dependencies to remove winapi and old windows-sys)
 - rust-lang/rust#146182 (Don't require next-solver `ProbeRef` to be `Copy`)

r? `@ghost`
`@rustbot` modify labels: rollup
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-compiletest Area: The compiletest test runner A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants