Nicolai Parlog
Nicolai Parlog / codefx.org / @nipafx
JUnit 5 is work in progress!
This is based on Milestone 3
(released 30th of November 2016).
Give feedback!
- GitHub repository
- User guide
- Articles on my blog
| Basics |
| Dynamic Tests |
| Extensions |
| Architecture |
| Tools & Setup |
class JUnit5Test {
@Test
void someTest() {
assertTrue(true);
}
}⇝ Package visibility suffices!
@BeforeAll
static void beforeAll() { ... }
@BeforeEach
void beforeEach() { ... }
@AfterEach
void afterEach() { ... }
@AfterAll
static void afterAll() { ... }⇝ Lifecycle annotations have new names.
@Test
@Disabled("Y U No Pass?!")
void failingTest() {
assertTrue(false);
}⇝ @Ignored is now @Disabled.
@Test
@DisabledOnFriday
void failingTest() {
assertTrue(false);
}⇝ Convenient Extensibility.
But how?
@Test
void someTest() {
...
assertEquals(
expected,
actual,
"Should be equal.");
}⇝ Failure message comes last.
@Test
void someTest() {
...
assertEquals(
expected,
actual,
() -> "Should " + "be " + "equal.");
}⇝ Failure message can be created lazily.
@Test
void assertAllProperties() {
Address ad = new Address(
"City", "Street", "42");
assertAll("address",
() -> assertEquals("C", ad.city),
() -> assertEquals("Str", ad.street),
() -> assertEquals("63", ad.number)
);
}⇝ assertAll gathers results from multiple assertions
Output if assertAll fails:
org.opentest4j.MultipleFailuresError:
address (3 failures)
expected: <C> but was: <City>
expected: <Str> but was: <Street>
expected: <63> but was: (42)void methodUnderTest() {
throw new IllegalStateException();
}
@Test
void assertExceptions() {
Exception ex = assertThrows(
Exception.class,
this::methodUnderTest);
assertEquals("Msg", ex.getMessage());
}⇝ assertThrows to assert
exception type and other properties
class CountTest {
// lifecycle and tests
@Nested
class CountGreaterZero {
// lifecycle and tests
@Nested
class CountMuchGreaterZero {
// lifecycle and tests
}
}
}⇝ @Nested to organize tests in inner classes
@DisplayName("A count")
class CountTest {
@Nested
@DisplayName("when greater zero")
class CountGreaterZero {
@Test
@DisplayName("is positive")
void isPositive() { ... }
}
}⇝ @DisplayName to show a nice name
@Test
void someTest(MyServer server) {
// do something with `server`
}⇝ Test has parameters!
But where do they come from?
| Basics |
| Dynamic Tests |
| Extensions |
| Architecture |
| Tools & Setup |
Up to now tests were identified
-
by names
(test…in JUnit 3 and before) -
by annotations
(@Testin JUnit 4 and 5)
⇝ Tests had to be known at compile time.
So what?!
What if we want to create tests
-
for a set of parameters
-
based on non-source files
-
with lambdas
⇝ We need to define tests at run time.
Allow creation of tests at run time.
-
tests are wrapped into
DynamicTest-s -
methods that create them
are annotated with@TestFactory
@TestFactory
List<DynamicTest> createPointTests() {
return Arrays.asList(
dynamicTest(
"A Great Test For Point",
() -> { /* test code */ } ),
dynamicTest(
"Another Great Test For Point",
() -> { /* test code */ } )
);
}The rest is straight-forward:
-
JUnit detects
@TestFactorymethods -
calls them to generate tests
-
adds tests to the test tree
-
eventually runs them
void pointTest(Point p) { /*...*/ }
@TestFactory
Stream<DynamicText> testingPoints() {
return Stream.of(
/* create points */)
.map(p ->
dynamicTest(
"Testing " + p,
() -> pointTest(p)));
}void pointTest(Point p) { /*...*/ }
@TestFactory
Stream<DynamicText> testingPoints() {
List<Point> points = asList(
/* create points */);
return DynamicTest.stream(
points,
p -> "Testing " + p,
p -> pointTest(p));
}void pointTest(Point p) { /*...*/ }
@TestFactory
Stream<DynamicText> testingPoints() {
return Files
.lines(pathToPointFile)
.map(Point::parse)
.map(p -> dynamicTest(
"Testing " + p,
() -> pointTest(p)));
}So what about lambdas?
This would be great:
class PointTest {
"A great test for point" -> {
/* test code */
}
}But how?
public class LambdaTest {
private List<DynamicTest> tests;
protected void λ(
String name, Executable test) {
tests.add(dynamicTest(name, test));
}
@TestFactory List<DynamicTest> tests() {
return tests;
}
}class PointTest extends LambdaTest {{
λ("A Great Test For Point", () -> {
/* test code goes here */
});
}}-
the inner braces create an initialization block
-
code therein is run during construction
-
JUnit will pick up the tests by calling
tests()
While we’re hacking… what about this?
class PointTest extends LambdaTest {{
λ(a_great_test_for_point -> {
/* test code goes here */
});
}}Learn how to access
a lambda’s parameter name
with
this one weird trick.
-
with
@TestFactoryandDynamicTest
we can create tests at run time:-
for sets of parameters
-
based on external input
-
as lambdas (yay!)
-
-
not fully integrated in lifecycle (yet? #378)
That's all very nice but is this already
Next Generation Testing?
| Basics |
| Dynamic Tests |
| Extensions |
| Architecture |
| Tools & Setup |
Manage a test’s full lifecycle.
@RunWith(MockitoJUnitRunner.class)
public class MyTest { ... }-
very flexible
-
heavyweight
-
exclusive
Execute code before and after statements.
public class MyTest {
@Rule
public MockitoRule rule =
MockitoJUnit.rule();
}-
added in 4.7
-
lightweight
-
limited to before/after behavior
Extension model is not optimal:
-
two competing mechanisms
-
each with limitations
-
but with considerable overlap
-
-
composition can cause problems
From JUnit 5’s Core Principles:
Prefer extension points over features
Quite literally,
JUnit 5 has Extension Points
-
Test Instance Post Processor
-
BeforeAll Callback
-
Test and Container Execution Condition
-
BeforeEach Callback
-
Parameter Resolution
-
Before Test Execution
-
After Test Execution
-
Exception Handling
-
AfterEach Callback
-
AfterAll Callback
-
one interface for each extension point
-
method arguments capture context
public interface BeforeEachCallback
extends Extension {
void beforeEach(
TestExtensionContext context);
}-
an extension can use multiple points
to implement its feature
We want to benchmark our tests!
-
for each test method
-
write the elapsed time to console
How?
-
before test execution: store test launch time
-
after test execution: print elapsed time
public class BenchmarkExtension implements
BeforeTestExecutionCallback,
AfterTestExecutionCallback {
private long launchTime;
// ...
}@Override
public void beforeTestExecution(
TestExtensionContext context) {
launchTime = currentTimeMillis();
}
@Override
public void afterTestExecution(
TestExtensionContext context) {
printf("Test '%s' took %d ms.%n",
context.getDisplayName(),
currentTimeMillis() - launchTime);
}Remember This?
@Test
@DisabledOnFriday
void failingTest() {
assertTrue(false);
}Let’s see how it works!
public class DisabledOnFridayCondition
implements TestExecutionCondition {
@Override
public ConditionEval.Result evaluate(
TestExtensionCtx. context) {
if (isFriday())
return disabled("Weekend!");
else
return enabled("Fix it!");
}
}What about parameter injection?
@Test
void someTest(MyServer server) {
// do something with `server`
}public class MyServerParameterResolver
implements ParameterResolver {
@Override
public boolean supports(
ParameterContext p, ...) {
return MyServer.class
== p.getParameter().getType();
}
@Override
public Object resolve( ... ) {
return new MyServer();
}
}Quick look at ExtensionContext:
// every node has its own context
Optional<ExtensionContext> getParent();
// some node-related info
String getUniqueId();
String getDisplayName();
Set<String> getTags();
// don't use System.out !
void publishReportEntry(
Map<String, String> map);Quick look at ExtensionContext:
// to reflect over the test class/method
Optional<AnnotatedElement> getElement();
Optional<Class<?>> getTestClass();
Optional<Method> getTestMethod();
// use the store to safe extension state
// (extensions should be stateless;
// did I mention that?)
Store getStore();
Store getStore(Namespace namespace);JUnit makes no promises regarding
extension instance lifecycle!
⇝ Extensions must be stateless!
Use the Store, Luke!
-
namespaced
-
hierarchical
-
key-value
Store is accessed via ExtensionContext
given a Namespace
// forwards with a default namespace
Store getStore();
Store getStore(Namespace namespace);-
keeps extensions from stepping
on each other’s toes -
could allow deliberate communication!
Reads from the store forward to other stores:
-
method store ⇝ class store
-
nested class store ⇝ surrounding class store
Writes always go to the called store.
The store is essentially a map:
Object getObject(Object key);
Object getOrComputeIfAbsent(
K key, Function creator);
void put(Object key, Object value)
Object remove(Object key)Overloads with type tokens exist.
void storeNowAsLaunchTime(
ExtensionContext context) {
long now = currentTimeMillis();
context.getStore(NAMESPACE)
.put(KEY, now);
}
long loadLaunchTime(
ExtensionContext context) {
return context.getStore(NAMESPACE)
.get(KEY, long.class);
}How do we apply extensions?
@ExtendWith(DisabledOnFridayCondition.class)
class JUnit5Test {
...
}That’s technical and verbose… :(
Meta-annotations to the rescue!
-
JUnit 5’s annotations are meta-annotations
-
JUnit 5 checks recursively for annotations
⇝ We can create our own annotations!
@ExtendWith(DisabledOnFridayCondition.class)
public @interface DisabledOnFriday { }
@Test
@Tag("integration")
@ExtendWith(BenchmarkExtension.class)
@ExtendWith(MyServerParameterResolver.class)
public @interface IntegrationTest { }
@IntegrationTest
@DisabledOnFriday
void testLogin(MyServer server) { ... }| Basics |
| Dynamic Tests |
| Extensions |
| Architecture |
| Tools & Setup |
-
a single JAR (ignoring Hamcrest)
-
used by
-
developers
-
extensions
-
IDEs, build-tools
-
-
no separation of concerns
-
tools provide us with awesome features!
-
but API is not powerful enough
I know, I’ll use reflection!
nothing was safe!
bound tools to implementation details
made maintenance and evolution very hard
Part of JUnit’s success is its great tool support!
But the same tools locked development in.
The success of JUnit as a platform prevents the development of JUnit as a tool.
(Johannes Link)
Separation of concerns:
-
an API to write tests against
-
a mechanism to discover and run tests
-
an API for tools to run tests
Separation of concerns V 2.0:
-
an API to write tests against
-
a mechanism to discover and run tests
-
specific engine per variant of tests
(e.g. JUnit 4 or JUnit 5) -
orchestration of engines
-
API between them
-
-
an API for tools to run tests
- JUnit Jupiter 5.0.0-M3
-
-
junit-jupiter-api -
junit-jupiter-engine
-
- JUnit Vintage 4.12.0-M3
-
-
junit-vintage-engine
-
- JUnit Platform 1.0.0-M3
-
-
junit-platform-engine -
junit-platform-runner -
a lot more
-
-
clear separation of concerns
-
API for developers
-
API for tools
That's all very nice but how is it
Next Generation Testing?
Because it opens up the platform!
-
want to run JUnit 4 tests?
⇝ create an engine for it -
want TestNG to have support like JUnit?
⇝ create an engine for it -
want to write tests in natural language?
⇝ create an engine for it
Once JUnit 5 adoption sets in:
-
tools are decoupled from implementation details
-
tools can support all frameworks (almost) equally well
-
new frameworks start with full tool support
-
developers can try out new things
A new generation of test frameworks might arise!
JUnit’s success as a platform
becomes available to everybody.
This heralds the
next generation of testing on the JVM!
-
clear separation of concerns:
APIs for developers, tools,
and new frameworks -
opens up the platform
-
tool support for everybody!
(There’s even more to the story.)
| Basics |
| Dynamic Tests |
| Extensions |
| Architecture |
| Tools & Setup |
-
individual classes:
@RunWith(JUnitPlatform.class) public class JUnit5Test { ... }
-
all classes:
@RunWith(JUnitPlatform.class) @SelectPackages({ "my.test.package" }) public class JUnit5TestSuite { }
JUnit 5 team provides rudimentary
Gradle plugin and Maven Surefire provider
(see user guide for details)
There is a console launcher:
# run all tests
junit-platform-console \
-cp ${path_to_compiled_test_classes} \
--scan-class-path
# run a specific test
junit-platform-console \
-cp ${path_to_compiled_test_classes} \
--select-class \
${fully_qualified_test_class_name}-
you can start writing tests right away
-
only IntelliJ has native support
-
running with JUnit 4 is a good compromise
(Read about the setup details.)
-
new API is an incremental improvement
full of thoughtful details -
dynamic tests are very useful
-
extension model looks very promising
-
architecture opens up the platform
-
tool support is not there yet
(Read more about JUnit 5.)
you can hire me
since 2016: editor of sitepoint.com/java
2014-2016: Java developer at Disy
2011-2014: Java developer at Fraunhofer ISI
until 2010: CS and Math at TU Dortmund
-
bubbles: Keith Williamson (CC-BY 2.0)
-
architecture diagrams:
Nicolai Parlog (CC-BY-NC 4.0) -
question-mark: Milos Milosevic (CC-BY 2.0)