TestNG Test Timeouts

While executing tests, there can be cases where certain tests get stuck or may take longer execution time than expected. In such scenarios, we may need to mark the said test as a fail and then move to the next test in the suite.

In this tutorial, we will learn to configure TestNG tests to timeout after some pre-configured duration.

1. Timeout Configuration

TestNG allows users to configure a time period to wait for a test to completely execute.

To specify timeout duration, use timeOut attribute of @Test annotation. The timeout value is in milliseconds.

@Test ( timeOut = 500 )

The timeout can be configured at two levels:

• Test Suite Level – This timeout will be applicable for all the tests in the said test suite. Each test in the suite must be executed with the configured timeout.
  The timeout counter will be reset after reach test method execution.
• Test Method Level – This timeout will be applicable for the said test method and will override the timeout period if configured at the suite level

Let’s create a sample test and learn how timeout works in TestNG.

2. Configuring Timeout for Test Suites

The timeout at suite level (that we can set in the testng.xml file) is not a timeout for each individual test, but a timeout for the whole suite. If we set the timeout to 10 minutes, then the suite will exit after 10 minutes, no matter if there are still tests left or not.

In the below example, we have two test methods i.e. timeTestOne() and timeTestTwo(). The timeTestOne() will take 1000ms to execute completely whereas timeTestTwo() will take 400ms to execute completely.

We have enforced the execution time using Thread.sleep() method.

public class TimeoutSuite
{
	@Test
	public void timeTestOne() throws InterruptedException {      //Fails
		Thread.sleep(1000);
		System.out.println("Time test method one");
	}

	@Test
	public void timeTestTwo() throws InterruptedException {      //Passes
		Thread.sleep(400);
		System.out.println("Time test method two");
	}
}

Now add a testng.xml file to the project root and put the following code to it. This code defines the timeout period as 500ms.

<suite name="Time test Suite" time-out="500" verbose="1" >
  <test name="Timeout Test" >
    <classes>
      <class name="com.howtodoinjava.test.TimeoutSuite" />
    </classes>
  </test>
</suite>

Now run the above tests using testng.xml. The output of the above test run is given below:

[TestNG] Running: C:\somepath\TestNGExamples\testng.xml

Time test method two

===============================================
Time test Suite
Total tests run: 2, Failures: 1, Skips: 0
===============================================

As you can see from the test results, only timeTestTwo() is passed because its execution time was less than the timeout time defined in testng.xml file. The timeTestOne() execution got failed because it took more time to complete than the timeout duration configured.

Let’s now go ahead and learn to set the timeout at a test method level.

3. Configuring Timeout for Test Methods

As mentioned earlier, you can specify the timeout at the individual test level as well. This will give you the flexibility to give appropriate time to run specific to each individual test method.

Note that timeout configuration at class level @Test annotation does not work. It works only at method level.

public class TimeoutMethod
{
	@Test(timeOut = 500)
	public void timeTestOne() throws InterruptedException {          //Fails
		Thread.sleep(1000);
		System.out.println("Time test method one");
	}

	@Test(timeOut = 500)
	public void timeTestTwo() throws InterruptedException {
		Thread.sleep(400);
		System.out.println("Time test method two");
	}
}

The output of the above test run is given below:

[[TestNG] Running: C:\Users\somepath\testng-customsuite.xml

Time test method two
PASSED: timeTestTwo
FAILED: timeTestOne

org.testng.internal.thread.ThreadTimeoutException: Method org.testng.internal.TestNGMethod.timeTestOne() didn't finish within the time-out 500

===============================================
    Default test
    Tests run: 2, Failures: 1, Skips: 0
===============================================
In above test methods timeTestOne() failed because it was not completely executed within timeout period specified.

Happy Learning !!

Sourcecode on Github