Optimized display of test results

[heisen-li]

Optimized display of test results

When trying to test using cargo:

1. When #[test] and #[bench] exist at the same time, the results of the other party will be included when cargo test/bench is executed. I think this shouldn't be mixed up and should be better presented.

2. I have been asked more than once where are the results of the unit test and where are the results of the integration test after executing the test. The current display content is not very clear. I think that in addition to adding a progress bar during the test , if there is a table summarizing the test after the test (including: unit test, Integration testing and documentation testing), that's pretty good.

3. In addition, are you considering adding the generated test report ./target/test_report? In this report, can increase the related reporting behavior of the test coverage.(like support: https://crates.io/crates/cargo-llvm-cov)

[epage]

For (1) and (2), it sounds like you are wanting hard lines between tests that are "unit" ,"integration", "doc test", etc. I'm not sure those distinctions are worth bubbling up to the top. I'd want to know more of why that is important.

In addition, are you considering adding the generated test report

Again, this seems to be solution driven. What problem are you looking to solve?

[heisen-li]

Frankly, I want to solve the display optimization of test results.

If there are many test cases, the overall test results will not be better displayed after executing the test.

Not only do I care about which test cases failed, I also want to know the overall results of document, unit, and integration tests through the command, likecargo test --all. If testing content such as benchmarks and test coverage are also included, my idea is that it can be opened in html. For example:

./target/testreport
             |——tests
                  | doc
                  | unit
                  | integration
                  | benchs
             |——test coverage（Include incremental testing）
             |——other

[epage]

Only a cursory explanation of the problem is given and still jumps straight to one specific solution. Please dive into the use cases and the problems with them so we can see what the best solutions are.

[heisen-li]

Maybe these two issues are consistent with my problem (issue1, issue2).

But in issue2, the advantage is that the summary display is very intuitive, but it is not clear which test cases failed.

Therefore, I think it is very convenient to use html format as one of the options for test report.

[epage]

One of the focuses of this effort will be that we shift rendering from libtest to cargo test by having a stable json output for libtest that cargo test will use to get results and decide how it wants to render it. That will make it trivial to fix both of those issues (don't show a lot of ceremony around test, save all of the existing summaries for the end).

Another outcome of this shift from libtest to cargo test is that it will allow other test runners to more easily exist and be able to extend cargo with more experimental functionality that, if appropriate, could one day move into cargo test proper.