JUnit 5 XML Report Example

In this JUnit 5 tutorial, we will learn to create XML reports in the newly supported Open Test Reporting format and generate them in legacy format for backward compatibility.

See Also: Generating HTML Reports with JUnit

1. Introduction

By default, the JUnit’s junit-platform-reporting artifact uses an implementation of TestExecutionListener that generates an XML report containing a test execution summary. Developers and 3rd-party tools can use this report for creating test execution reports in various formats.

Currently, JUnit 5 supports XML report creation in two formats:

• Legacy reporting
• Open Test Reporting (added in JUnit 5.9.x)

Open Test Reporting is a collection of new test reporting formats, agnostic testing frameworks and programming languages. There are currently two formats in the specification.

• The first format is event-based, suitable for writing events to a file and streaming events over a local socket or network connection. JUnit 5 uses this format.
• The second format is similar to existing hierarchical representations of test results that present the execution result with a tree of tests and their results. Legacy JUnit reports are in this format.

2. JUnit XML reports in Open Test Reporting format

2.1. Configuration

To create the test execution report in the new format, ensure you have the latest version of junit-platform-reporting dependency in the project.

<dependency>
  <groupId>org.junit.platform</groupId>
  <artifactId>junit-platform-reporting</artifactId>
  <version>1.9.1</version>
  <scope>test</scope>
</dependency>

dependencies {
    testRuntimeOnly("org.junit.platform:junit-platform-reporting:1.9.1")
}

Next, we must pass the following parameters to enable the new or legacy reporting formats. The first parameter enables/disables writing the report. The second parameter configures the output directory.

By default, the output directory is set to '/build‘ if a Gradle build script is found and '/target' if a Maven POM is found; otherwise, the current working directory is used.

• junit.platform.reporting.open.xml.enabled = true/false
• junit.platform.reporting.output.dir = <path>

The followings are the sample config entries for build systems:

<build>
	<plugins>
		<plugin>
			<artifactId>maven-surefire-plugin</artifactId>
			<version>2.22.2</version>
			<configuration>
				<properties>
					<configurationParameters>
						junit.platform.reporting.open.xml.enabled = true
						junit.platform.reporting.output.dir = target/surefire-reports
					</configurationParameters>
				</properties>
			</configuration>
		</plugin>
	</plugins>
</build>

tasks.withType(Test).configureEach {
	def outputDir = reports.junitXml.outputLocation
	jvmArgumentProviders << ({
		[
			"-Djunit.platform.reporting.open.xml.enabled=true",
			"-Djunit.platform.reporting.output.dir=${outputDir.get().asFile.absolutePath}"
		]
	} as CommandLineArgumentProvider)
}

2.2. XML Report

Now when we run the 'mvn test' command, the OpenTestReportGeneratingListener creates an XML report file named junit-platform-events-random_id.xml per test run in the configured output directory.

For demo purposes, let us run the following test class that has two tests. The first test succeeds and the second test fails.

public class HelloTest {
  @Test
  void testOne(){
    Assertions.assertTrue(true);
  }

  @Test
  void testTwo(){
    Assertions.assertTrue(false);
  }
}

Run the test from the console.

$ mvn test

And check out the generated XML report file with name: junit-platform-events-3100202407917663308.xml.

<?xml version="1.0" ?>
<e:events ...>
	<infrastructure>
		....
	</infrastructure>
	<e:started id="1" name="JUnit Jupiter" time="2022-10-07T10:57:29.193634500Z">
		<metadata>
			<junit:uniqueId>[engine:junit-jupiter]</junit:uniqueId>
			<junit:legacyReportingName>JUnit Jupiter</junit:legacyReportingName>
			<junit:type>CONTAINER</junit:type>
		</metadata>
	</e:started>
	<e:started id="2" name="HelloTest" parentId="1" time="2022-10-07T10:57:29.247574500Z">
		<metadata>
			<junit:uniqueId>[engine:junit-jupiter]/[class:com.howtodoinjava.junit5.examples.xmlReport.HelloTest]</junit:uniqueId>
			<junit:legacyReportingName>com.howtodoinjava.junit5.examples.xmlReport.HelloTest</junit:legacyReportingName>
			<junit:type>CONTAINER</junit:type>
		</metadata>
		<sources>
			<java:classSource className="com.howtodoinjava.junit5.examples.xmlReport.HelloTest"/>
		</sources>
	</e:started>
	<e:started id="3" name="testOne()" parentId="2" time="2022-10-07T10:57:29.269523200Z">
		<metadata>
			<junit:uniqueId>[engine:junit-jupiter]/[class:com.howtodoinjava.junit5.examples.xmlReport.HelloTest]/[method:testOne()]</junit:uniqueId>
			<junit:legacyReportingName>testOne()</junit:legacyReportingName>
			<junit:type>TEST</junit:type>
		</metadata>
		<sources>
			<java:methodSource className="com.howtodoinjava.junit5.examples.xmlReport.HelloTest" methodName="testOne" methodParameterTypes=""/>
		</sources>
	</e:started>
	<e:finished id="3" time="2022-10-07T10:57:29.307527100Z">
		<result status="SUCCESSFUL"/>
	</e:finished>
	<e:started id="4" name="testTwo()" parentId="2" time="2022-10-07T10:57:29.312537Z">
		<metadata>
			<junit:uniqueId>[engine:junit-jupiter]/[class:com.howtodoinjava.junit5.examples.xmlReport.HelloTest]/[method:testTwo()]</junit:uniqueId>
			<junit:legacyReportingName>testTwo()</junit:legacyReportingName>
			<junit:type>TEST</junit:type>
		</metadata>
		<sources>
			<java:methodSource className="com.howtodoinjava.junit5.examples.xmlReport.HelloTest" methodName="testTwo" methodParameterTypes=""/>
		</sources>
	</e:started>
	<e:finished id="4" time="2022-10-07T10:57:29.317529600Z">
		<result status="FAILED">
			<java:throwable assertionError="true" type="org.opentest4j.AssertionFailedError">
				<![CDATA[org.opentest4j.AssertionFailedError: expected: <true>but was: <false>at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151) ...]]>
			</java:throwable>
		</result>
	</e:finished>
	<e:finished id="2" time="2022-10-07T10:57:29.325526800Z">
		<result status="SUCCESSFUL"/>
	</e:finished>
	<e:finished id="1" time="2022-10-07T10:57:29.331528600Z">
		<result status="SUCCESSFUL"/>
	</e:finished>
	<e:started id="5" name="JUnit Platform Suite" time="2022-10-07T10:57:29.332527800Z">
		<metadata>
			<junit:uniqueId>[engine:junit-platform-suite]</junit:uniqueId>
			<junit:legacyReportingName>JUnit Platform Suite</junit:legacyReportingName>
			<junit:type>CONTAINER</junit:type>
		</metadata>
	</e:started>
	<e:finished id="5" time="2022-10-07T10:57:29.333523400Z">
		<result status="SUCCESSFUL"/>
	</e:finished>
	<e:started id="6" name="JUnit Vintage" time="2022-10-07T10:57:29.333523400Z">
		<metadata>
			<junit:uniqueId>[engine:junit-vintage]</junit:uniqueId>
			<junit:legacyReportingName>JUnit Vintage</junit:legacyReportingName>
			<junit:type>CONTAINER</junit:type>
		</metadata>
	</e:started>
	<e:finished id="6" time="2022-10-07T10:57:29.335541600Z">
		<result status="SUCCESSFUL"/>
	</e:finished>
</e:events>

3. JUnit XML reports in legacy format

The easiest way to generate the legacy XML report is to set the parameter junit.platform.reporting.open.xml.enabled to false.

junit.platform.reporting.open.xml.enabled = false

When we rerun the test, LegacyXmlReportGeneratingListener generates the default report in the following format with the name TEST-com.howtodoinjava.junit5.examples.xmlReport.HelloTest.xml.

<?xml version="1.0" encoding="UTF-8"?>
<testsuite errors="0" failures="1" name="com.howtodoinjava.junit5.examples.xmlReport.HelloTest" 
	skipped="0" tests="2" time="0.076" version="3.0" ...>
	<properties>
		<property name="java.specification.version" value="17"/>
		<property name="sun.cpu.isalist" value="amd64"/>
		<property name="sun.jnu.encoding" value="Cp1252"/>
		<property name="java.class.path" value="..."/>
		<property name="java.vm.vendor" value="Oracle Corporation"/>
		...
	</properties>
	<testcase classname="com.howtodoinjava.junit5.examples.xmlReport.HelloTest" name="testOne" time="0.04"/>
	<testcase classname="com.howtodoinjava.junit5.examples.xmlReport.HelloTest" name="testTwo" time="0.008">
		<failure message="expected: &lt;true&gt; but was: &lt;false&gt;" type="org.opentest4j.AssertionFailedError">
			<![CDATA[org.opentest4j.AssertionFailedError: expected: <true>but was: <false>at 
			org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)...]]>
		</failure>
	</testcase>
</testsuite>

4. Conclusion

This short tutorial taught us about JUnit’s XML report generation in legacy and new open-test reporting formats. Also, we learned to switch a specific version using the configuration parameters, including the output directory location of the report.

Happy Learning !!