Description of the bug
When running nf-core modules test <tool> I ran into the issue that all tests ran succesfully, but I got an error message:
CRITICAL Ran, but found errors:
- nf-test failed
- nf-test failed
Looking into the code, I found that in the display_nftest_output function of the tests the if-statement seems incorrect. If the "Different Snapshot" is not in the nftest_err message, it will always print the "nf-test failed" message. So when you have no error message (all tests succesfull), it prints that.
Additionally, if you run nf-core modules test <tool> --no-prompts you go into the top part of the if-statement and I correctly see that all tests passed, so it is only a problem if you run without --no-prompts
def display_nftest_output(self, nftest_out: bytes, nftest_err: bytes) -> None:
if self.no_prompts:
# In no_prompts mode (e.g., tests), use simple logging to avoid Rich hanging in non-TTY environments
log.debug("nf-test output:\n%s", nftest_out.decode())
if nftest_err:
log.debug("nf-test error:\n%s", nftest_err.decode())
if "Different Snapshot:" in nftest_err.decode() and self.update:
log.info("Updating snapshot")
self.generate_snapshot()
else:
# Interactive mode: use Rich formatting
print("Displaying nf-test output")
nftest_output = Text.from_ansi(nftest_out.decode())
print(Panel(nftest_output, title="nf-test output"))
print("Displaying nf-test error", nftest_err.decode())
if nftest_err:
print("Parsing nf-test error")
syntax = Syntax(nftest_err.decode(), "diff", theme="ansi_dark")
print("Parsed nf-test error")
print(Panel(syntax, title="nf-test error"))
print("Displaying nf-test error")
if "Different Snapshot:" in nftest_err.decode():
log.error("nf-test failed due to differences in the snapshots")
if self.update is None:
answer = Confirm.ask(
"[bold][blue]?[/] nf-test found differences in the snapshot. Do you want to update it?",
default=True,
)
if answer:
log.info("Updating snapshot")
self.update = True
else:
log.debug("Snapshot not updated")
if self.update:
# update snapshot using nf-test --update-snapshot
self.generate_snapshot()
else:
self.errors.append("nf-test failed")
Command used and terminal output
nf-core modules test nasvar/pipeline --profile docker
,--./,-.
___ __ __ __ ___ /,-._.--~\
|\ | |__ __ / ` / \ |__) |__ } {
| \| | \__, \__/ | \ |___ \`-._,-`-,
`._,._,'
nf-core/tools version 4.0.3 - https://nf-co.re
INFO Generating nf-test snapshot
Displaying nf-test output
╭────────────────────────────────────────────────────────────────────────────────────────────────────── nf-test output ───────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ 🚀 nf-test 0.9.5 │
│ https://www.nf-test.com │
│ Please cite: https://doi.org/10.1093/gigascience/giaf130 │
│ (c) 2021 - 2026 Lukas Forer and Sebastian Schoenherr │
│ │
│ Load .nf-test/plugins/nft-anndata/0.4.1/nft-anndata-0.4.1.jar │
│ Load .nf-test/plugins/nft-bam/0.6.1/nft-bam-0.6.1.jar │
│ Load .nf-test/plugins/nft-csv/0.1.0/nft-csv-0.1.0.jar │
│ Load .nf-test/plugins/nft-compress/0.1.0/nft-compress-0.1.0.jar │
│ Load .nf-test/plugins/nft-fastq/0.1.0/nft-fastq-0.1.0.jar │
│ Load .nf-test/plugins/nft-utils/0.0.9/nft-utils-0.0.9.jar │
│ Load .nf-test/plugins/nft-vcf/1.0.7/nft-vcf-1.0.7.jar │
│ │
│ Test Process NASVAR_PIPELINE │
│ │
│ Test [f427fc01] 'nasvar pipeline - bam - RUNX1::ETV6' PASSED (150.613s) │
│ Test [9a2062a3] 'nasvar pipeline - bam - stub' PASSED (3.621s) │
│ │
│ │
│ SUCCESS: Executed 2 tests in 154.287s │
│ │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Displaying nf-test error
INFO Generating nf-test snapshot again to check stability
Displaying nf-test output
╭────────────────────────────────────────────────────────────────────────────────────────────────────── nf-test output ───────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ 🚀 nf-test 0.9.5 │
│ https://www.nf-test.com │
│ Please cite: https://doi.org/10.1093/gigascience/giaf130 │
│ (c) 2021 - 2026 Lukas Forer and Sebastian Schoenherr │
│ │
│ Load .nf-test/plugins/nft-anndata/0.4.1/nft-anndata-0.4.1.jar │
│ Load .nf-test/plugins/nft-bam/0.6.1/nft-bam-0.6.1.jar │
│ Load .nf-test/plugins/nft-csv/0.1.0/nft-csv-0.1.0.jar │
│ Load .nf-test/plugins/nft-compress/0.1.0/nft-compress-0.1.0.jar │
│ Load .nf-test/plugins/nft-fastq/0.1.0/nft-fastq-0.1.0.jar │
│ Load .nf-test/plugins/nft-utils/0.0.9/nft-utils-0.0.9.jar │
│ Load .nf-test/plugins/nft-vcf/1.0.7/nft-vcf-1.0.7.jar │
│ │
│ Test Process NASVAR_PIPELINE │
│ │
│ Test [f427fc01] 'nasvar pipeline - bam - RUNX1::ETV6' PASSED (143.381s) │
│ Test [9a2062a3] 'nasvar pipeline - bam - stub' PASSED (3.259s) │
│ │
│ │
│ SUCCESS: Executed 2 tests in 146.689s │
│ │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Displaying nf-test error
CRITICAL Ran, but found errors:
- nf-test failed
- nf-test failed
(.venv) (base) k.v.cammel@MB-GXG4Q2Q5R6 modules % nf-core modules test nasvar/pipeline --profile docker --no-prompts
,--./,-.
___ __ __ __ ___ /,-._.--~\
|\ | |__ __ / ` / \ |__) |__ } {
| \| | \__, \__/ | \ |___ \`-._,-`-,
`._,._,'
nf-core/tools version 4.0.3 - https://nf-co.re
INFO Generating nf-test snapshot
INFO Generating nf-test snapshot again to check stability
INFO All tests passed!System information
Nextflow version 26.04
Nf-test 0.9.5
nf-core 4.0.2 & 4.0.3 (also tried it with the dev version)
Ran on macbook with --profile docker
Description of the bug
When running
nf-core modules test <tool>I ran into the issue that all tests ran succesfully, but I got an error message:Looking into the code, I found that in the display_nftest_output function of the tests the if-statement seems incorrect. If the "Different Snapshot" is not in the nftest_err message, it will always print the "nf-test failed" message. So when you have no error message (all tests succesfull), it prints that.
Additionally, if you run
nf-core modules test <tool> --no-promptsyou go into the top part of the if-statement and I correctly see that all tests passed, so it is only a problem if you run without--no-promptsCommand used and terminal output
System information
Nextflow version 26.04
Nf-test 0.9.5
nf-core 4.0.2 & 4.0.3 (also tried it with the dev version)
Ran on macbook with --profile docker