In JUnit 5, writing ordered tests is highly discouraged, there may be situations where test execution ordering becomes an important factor for testing the application functionality such as integration testing.
For example, we may want the integration testing for a user management module where the tests should execute in the following manner: create a new user, update the user and then delete the user.
Read More : Test ordering in JUnit 4
1. Test Methods Ordering
JUnit 5 (checked with 5.8.1) supports the following ways to order tests in the interface MethodOrderer
:
- MethodName – sorts tests alphanumerically based on their names.
- DisplayName – sorts tests alphanumerically based on their display names.
- OrderAnnotation – sorts tests based on the @Order annotation. Tests with same order will be sorted arbitrarily adjacent to each other. Not ordered tests will appear at the end.
- Random – orders tests pseudo-randomly. Randomness of tests in each execution can be controlled by property
junit.jupiter.execution.order.random.seed
. - Custom – orders the test with provided ordering logic.
Another ordering class Alphanumeric has been marked deprecated and will be removed in JUnit 6. So you can avoid using it.
1.1. MethodOrderer.MethodName
An example of sorting the tests based on their method names.
@TestMethodOrder(MethodOrderer.MethodName.class)
public class MethodNameOrderedTests
{
@Test
void testE() {
assertTrue(true);
}
@Test
void testA() {
assertTrue(true);
}
@Test
void testD() {
assertTrue(true);
}
@Test
void testC() {
assertTrue(true);
}
@Test
void testB() {
assertTrue(true);
}
}
testA()
testB()
testC()
testD()
testE()
1.2. MethodOrderer.DisplayName
An example of sorting the tests based on their display names.
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
@TestMethodOrder(MethodOrderer.DisplayName.class)
public class DisplayNameOrderedTests
{
@DisplayName("5")
@Test
void testE() {
assertTrue(true);
}
@DisplayName("3")
@Test
void testA() {
assertTrue(true);
}
@DisplayName("1")
@Test
void testD() {
assertTrue(true);
}
@DisplayName("2")
@Test
void testC() {
assertTrue(true);
}
@DisplayName("4")
@Test
void testB() {
assertTrue(true);
}
}
1
2
3
4
5
1.3. MethodOrderer.OrderAnnotation
An example of sorting the tests based on their ordering as mentioned in @Order
annotation.
import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Order;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class OrderAnnotationTests {
@Order(5)
@Test
void testE() {
assertTrue(true);
}
@Order(4)
@Test
void testA() {
assertTrue(true);
}
@Order(3)
@Test
void testD() {
assertTrue(true);
}
@Order(2)
@Test
void testC() {
assertTrue(true);
}
@Order(1)
@Test
void testB() {
assertTrue(true);
}
}
testB()
testC()
testD()
testA()
testE()
1.4. MethodOrderer.Random
An example of sorting the tests randomly. By default, the random seed used for ordering methods is the System.nanoTime()
which is generated during the static initialization of the test class.
We can change the randomness by providing a custom seed through property junit.jupiter.execution.order.random.seed.
junit.jupiter.execution.order.random.seed=9999
@TestMethodOrder(MethodOrderer.Random.class)
public class RandomOrderedTests {
@Test
void testE() {
assertTrue(true);
}
@Test
void testA() {
assertTrue(true);
}
@Test
void testD() {
assertTrue(true);
}
@Test
void testC() {
assertTrue(true);
}
@Test
void testB() {
assertTrue(true);
}
}
By default, the output will be different for each execution of the test class. But if we use a fixed seed using the property files, the order of the tests will be fixed and will not change between different executions.
The given output is after we have specified the custom seed 9999.
//Execution 1
testB()
testC()
testE()
testA()
testD()
//Execution 2
testB()
testC()
testE()
testA()
testD()
1.5. Custom Ordering
It is possible to define your own custom ordering by implementing the interface MethodOrderer
.
In the given example, we are executing the deprecated tests in the end.
import java.util.Comparator;
import org.junit.jupiter.api.MethodDescriptor;
import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.MethodOrdererContext;
public class DeprecatedInEndTestOrder implements MethodOrderer {
private Comparator<MethodDescriptor> comparator = Comparator
.comparing(md -> md.getMethod().isAnnotationPresent(Deprecated.class));
@Override
public void orderMethods(MethodOrdererContext context) {
context.getMethodDescriptors().sort(comparator);
}
}
import static org.junit.Assert.assertTrue;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
@TestMethodOrder(DeprecatedInEndTestOrder.class)
public class CustomOrderTests {
@Test
@Deprecated
void testC() {
assertTrue(true);
}
@Test
void testA() {
assertTrue(true);
}
@Test
void testD() {
assertTrue(true);
}
@Deprecated
@Test
void testE() {
assertTrue(true);
}
@Test
void testB() {
assertTrue(true);
}
}
Notice the test output. The deprecated tests have been executed in the last.
testA()
testB()
testD()
testC()
testE()
2. Test Classes Ordering
Reiterating the fact that test classes typically should not rely on the execution order, there are times when it may be needed.
2.1. Supported Ordering Types
JUnit 5 (checked with 5.8.1) supports the following ways to order tests in the interface ClassOrderer. We can use the class order in the same way as method ordering seen in the above sections.
- ClassName – sorts test classes alphanumerically based on their fully qualified class names.
- DisplayName – sorts test classes alphanumerically based on their display names.
- OrderAnnotation – sorts test classes numerically based on values specified via the @Order annotation.
- Random – orders test classes pseudo-randomly. Supports custom seed using the property junit.jupiter.execution.order.random.seed.
To set the order globally for all test classes, use the configuration property junit.jupiter.testclass.order.default
.
junit.jupiter.testclass.order.default = org.junit.jupiter.api.ClassOrderer$OrderAnnotation
2.2. Test Class Ordering Example
Given is an example for test class ordering in nested test classes.
@TestClassOrder(ClassOrderer.OrderAnnotation.class)
class OrderedTestClassesExample {
@Nested
@Order(1)
class SetupTests {
@Test
void test1() {
}
}
@Nested
@Order(2)
class AppFlowTests {
@Test
void test2() {
}
}
}
SetupTests
- test1()
AppFlowTests
- test2()
3. Conclusion
Let me iterate again that forcing an ordering on the test execution order is not recommended and should be avoided. Still, if you come across such a requirement then use above explained techniques to order the test classes and test methods.
I will recommend using @Order annotation with ClassOrderer.OrderAnnotation and MethodOrderer.OrderAnnotation to have full control over the ordering of tests if the test ordering is such important in your case.
Happy Learning !!