surefire 3.6.0: JUnit Platform provider silently fails to discover tests when package name contains the substring "java"

[ZiadMarey]

Affected version

3.6.0 (regression — works correctly on 3.5.6)

Bug description

maven-surefire-plugin/maven-failsafe-plugin 3.6.0's JUnit Platform provider silently fails to discover any test in a module whose package name contains the substring java as a whole path segment (e.g. com.example.its.java.SomeIT) — but only when the module's project layout matches a specific combination described below. The build reports BUILD SUCCESS with Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 — no error, no warning, no exception anywhere in -X debug output. The affected class's name never even appears in the log (no Running com.example... line is printed, meaning discovery drops it before execution is attempted at all).

Reproduction

We confirmed this with a clean, controlled A/B test in our own multi-module project:

• A trivial test class com.example.its.probe.ProbeIT (package without "java") → discovered and runs correctly (Tests run: 1).
• The exact same trivial class, only its package renamed to com.example.its.javaprobe.ProbeIT (package containing "java" as a substring) → Tests run: 0, never discovered, in the same module/build/classpath.
• Repeated with a real, substantive, pre-existing test class (not a toy example): identical file, only the package changed from ...its.java to ...its.probe — same result. In ...its.java it's never discovered; the identical file relocated to ...its.probe is discovered and actually runs.

Downgrading only maven-surefire-plugin/maven-failsafe-plugin to 3.5.6 for the affected module (via a module-level plugin version override, <pluginManagement> untouched) immediately fixes discovery, with no other change.

What we ruled out

Since this symptom looks similar to #3466 (fixed) / #3474 (still open), we suspected the same "class name matched against a file-path-style pattern" bug class. However:

• Our project's failsafe config does not use %regex[...] includes anywhere.
• We tried to reproduce the divergence by calling the real compiled 3.6.0 classes directly (TestListResolver.shouldRun, and a faithful port of JUnitPlatformProvider.matchClassName using the real SelectorUtils) with our exact class names — neither showed any difference in behavior between a "java"-containing package and one without, in isolation. So while the symptom matches the known bug class, we were not able to pin down the exact line responsible, and this may be a distinct (or interacting) trigger condition.
• We also ruled out (by direct testing): module-specific dependencies, base class hierarchies, excludedGroups/groups configuration, CI runner type, Maven cache state, and git submodule initialization steps — none of these affect the reproduction.

Environment

• maven-surefire-plugin / maven-failsafe-plugin: 3.6.0
• JUnit Jupiter: 5.14.4 / JUnit Platform: 1.14.4
• JDK 21
• Reproduced on both Windows and Linux (GitHub Actions ubuntu-based runners)
• Multi-module Maven reactor (the affected module is one of several sibling modules with an otherwise near-identical layout; only the one whose package contains "java" is affected)

Happy to provide a minimal standalone reproducer repository if useful.

[orion0616]

@ZiadMarey
I authored #3466, so I checked how 3.6.0 handles a package containing "java".

JUnitPlatformProvider#newFilters removes every occurrence of .java from each non-regex include/exclude pattern, not just the suffix (StringUtils.replace(pattern, ".java", "")). This happens before matchClassName, which could explain why your port did not reproduce it. A pattern containing a dotted class name gets mangled:

pattern	after replacement
com.example.its.java.SomeIT	com.example.its.SomeIT
com.example.its.javaprobe.ProbeIT	com.example.itsprobe.ProbeIT
com.example.its.probe.ProbeIT	unchanged

That would match your split. A slash-style pattern such as com/example/its/java/*IT.java keeps its package intact under this replacement.

Could you share the affected module's effective <includes>/<excludes>, any <includesFile>/<excludesFile> contents, and the exact command line, including any -Dit.test or -Dtest?

If a dotted pattern containing .java is involved, this could be the cause. #3466 is merged into master (milestone 3.6.1) and removes that replacement entirely. Testing your project with a local build of master would help confirm whether it resolves your case.

[ZiadMarey]

Thanks for the detailed explanation — that matches our findings exactly, and explains precisely why our attempt to reproduce it by directly calling matchClassName in isolation didn't show the divergence (we never applied the .java-stripping preprocessing step to the pattern first).

Answering your questions:

Effective <includes>/<excludes> for the affected module:

• No custom <includes>/<excludes> are configured for the execution that fails (bound to the integration-test goal) — it relies entirely on failsafe's built-in default patterns (**/IT*.java, **/*IT.java, **/*ITCase.java).
• A separate execution in the same module does set an explicit include (**/*E2E*.java), but that one is unaffected since none of our failing classes match it.
• No <includesFile>/<excludesFile> are used anywhere in the project.

Exact command line:

mvn verify -Dit.test=com.example.its.java.SomeIT -DskipITs=false ...

i.e. a single fully-qualified class name (sometimes with #methodName), passed via -Dit.test, no wildcards or regex on our end.

Confirmation against your fix: we built maven-surefire-plugin/maven-failsafe-plugin from your master branch locally (installed as 3.6.1-SNAPSHOT) and re-ran our affected module against it. Discovery now works correctly — every test class we tried was found and attempted, where before all of them silently returned Tests run: 0:

Running com.example.its.java.SomeAIT       -> Tests run: 1
Running com.example.its.java.SomeBIT       -> Tests run: 1
Running com.example.its.java.SomeCIT       -> Tests run: 1
Running com.example.its.java.SomeDIT       -> Tests run: 1
Running com.example.its.java.SomeEIT       -> Tests run: 1

(Each subsequently hit an unrelated, pre-existing local environment issue on our end, which is expected and unrelated to discovery — the point is that discovery itself now succeeds for all of them, where it previously failed for all of them.)

This confirms #3466 fixes our case. Looking forward to the 3.6.1 release — thank you for the quick and precise diagnosis.