Grouping tests is one of the most important features of TestNG. In TestNG, users can group multiple tests into a named group. The user can execute a particular set of tests belonging to a group or multiple groups.
Test grouping feature allows the tests to be segregated into different sections or modules. For example, we can have a set of tests that belong to the sanity tests whereas others may belong to regression tests.
We can also segregate the tests based on the functionalities/features that the test verifies. This helps in executing only a particular set of tests as and when required.
In this tutorial, we’ll learn about the grouping of tests in TestNG in below sections.
Table of Contents 1. Creating test groups 2. Running a TestNG group through Eclipse 3. Running a TestNG group through testng.xml 4. Writing tests which belong to multiple groups 5. Including and excluding groups 6. Using regular expressions 7. Default group 8. Group of groups
1. Creating test groups
Let’s create a test class, which contains certain tests that belong to a group. To do this, use groups
attribute of @Test
annotation.
package com.howtodoinjava.groupExamples; import org.testng.annotations.Test; public class TestGroupExample { @Test(groups = { "test-group" }) public void testMethodOne() { System.out.println("Test method one belonging to group."); } @Test public void testMethodTwo() { System.out.println("Test method two not belonging to group."); } @Test(groups = { "test-group" }) public void testMethodThree() { System.out.println("Test method three belonging to group."); } }
If you will run the above test in eclipse normally, then test execution does not consider the group for execution and hence executes all the tests in the specified test class.
If you want to execute methods under a certain group only, then you will execute them in either one manner discussed in the following two sections.
2. Running groups through Eclipse
In the earlier section, we created a test class with certain tests that belonged to a test group. Now let’s run the group of tests using eclipse.
- Go to Run | Run Configurations.
- Select TestNG from the list of available configurations and click on the new configuration icon.
- In the new configuration window give a configuration name, for example, TestGroupExample.
- Go to the Project section and click on the Browse button. Select the previously created project that is TestNGExamples.
-
Go to the Groups section and click on the Browse button. Select the group which you would like to execute from the list, in this case it’s test-group.
Click on the Apply
button and then click on Run
.
The following results will be shown in the TestNG’s Results window of Eclipse:
Test method one belonging to group. Test method three belonging to group. PASSED: testMethodOne PASSED: testMethodThree =============================================== GRP-test-group Tests run: 2, Failures: 0, Skips: 0 ===============================================
Congratulations, you have successfully executed test methods that belonged to a particular group using the TestNG runner configuration in Eclipse. You can also use the utility to execute multiple groups by selecting the respective groups in the Browse section.
Normally it’s better to use the TestNG-XML-based execution to execute test methods that belong to a particular group.
3. Running groups through testng.xml
Now let’s learn how to create a TestNG XML file to execute tests that belong to a particular group. This method is the preferred and easy way to execute groups. Also, these TestNG XML files can then be used with the build tools to execute TestNG test suites.
- Open Eclipse and create a new file with the name
testng.xml
in the previously created project. - Add the following code to the said file:
<suite name="Time test Suite" verbose="1"> <test name="Group Test"> <groups> <run> <include name="test-group" /> </run> </groups> <classes> <class name="com.howtodoinjava.groupExamples.TestGroupExample" /> </classes> </test> </suite>
This XML file contains only one test inside a suite. This contains the groups section defined by using the
groups
tag as shown in the code. Therun
tag represents the group that needs to be run. Theinclude
tag represents the name of the group that needs to be executed. - Select the previously created testng XML file and run it as a TestNG suite. You will see the following test results:
Test method one belonging to group. Test method three belonging to group. =============================================== Time test Suite Total tests run: 2, Failures: 0, Skips: 0 ===============================================
Great. We successfully created a testng XML file that creates a test in the said suite by including a group in it. This is done by including the said group inside the run section. The run section is, in turn, part of the groups tag section inside the test.
TestNG will look for test methods that belong to the said group under the class that is mentioned in the classes section of the test.
The user can also provide packages for the tests. TestNG will search all the classes that are added to the test to include or exclude particular test methods that belong to particular groups. Once found, these test methods will then be executed by TestNG as a test suite.
4. Writing tests to multiple groups
Earlier we learned about creating tests that belonged to a single group, but TestNG allows test methods to belong to multiple groups also. This can be done by providing the group names as an array in the groups attribute of the @Test
annotation.
Let’s create a sample program with multiple groups to learn how it is done.
package com.howtodoinjava.groupExamples; import org.testng.annotations.Test; public class MultiGroupExample { @Test(groups = { "group-one" }) public void testMethodOne() { System.out.println("Test method one belonging to group."); } @Test(groups = { "group-one", "group-two" }) public void testMethodTwo() { System.out.println("Test method two belonging to both group."); } @Test(groups = { "group-two" }) public void testMethodThree() { System.out.println("Test method three belonging to group."); } }
The preceding class contains three test methods. Two of the test methods belong to one group each, whereas one of the methods belongs to two groups, group-one
, and group-two
respectively.
Now edit the testng.xml
file like below:
<suite name="Multi Group Suite" verbose="1"> <test name="Group Test one"> <groups> <run> <include name="group-one" /> </run> </groups> <classes> <class name="com.howtodoinjava.groupExamples.MultiGroupExample" /> </classes> </test> <test name="Group Test two"> <groups> <run> <include name="group-two" /> </run> </groups> <classes> <class name="com.howtodoinjava.groupExamples.MultiGroupExample" /> </classes> </test> </suite>
The preceding TestNG XML suite contains two tests, each of them executing test methods belonging to a particular group. Select TestNG XML file and run it as a TestNG suite. You will see the following test results:
Test method one belonging to group. Test method two belonging to both group. Test method three belonging to group. Test method two belonging to both group. =============================================== Multi Group Suite Total tests run: 4, Failures: 0, Skips: 0 ===============================================
Here we have successfully created a test method, which belongs to multiple groups and can be executed successfully. As you can see in the previous test result, testMethodTwo()
was executed in both the tests of the test suite. This is because it belongs to both of the groups whose test methods are executed by TestNG.
5. Including and excluding groups from suite
TestNG also allows you to include and exclude certain groups from test execution. This helps in executing only a particular set of tests and excluding certain tests.
A simple example can be when a feature is broken and you need to exclude a fixed set of tests from execution since these tests will fail upon execution. Once the feature is fixed you can then verify the feature by just executing the respective group of tests.
Let’s create a sample program and learn how to exclude a group of tests.
package com.howtodoinjava.groupExamples; import org.testng.annotations.Test; public class ExcludeGroupTest { @Test(groups = { "include-group" }) public void testMethodOne() { System.out.println("Test method one belonging to include group."); } @Test(groups = { "include-group" }) public void testMethodTwo() { System.out.println("Test method two belonging to include group."); } @Test(groups = { "include-group", "exclude-group" }) public void testMethodThree() { System.out.println("Test method three belonging to exclude/include groups."); } }
The preceding class contains three test methods that print a message onto console when executed. All the three methods belong to a group include-group
whereas the testMethodThree()
method also belongs to the group exclude-group
.
<suite name="Exclude Group Suite" verbose="1"> <test name="Exclude Group Test"> <groups> <run> <include name="include-group" /> <exclude name="exclude-group" /> </run> </groups> <classes> <class name="com.howtodoinjava.groupExamples.ExcludeGroupTest" /> </classes> </test> </suite>
Now run above testng.xml
file and it will produce the below result.
Test method one belonging to include group. Test method two belonging to include group. =============================================== Exclude Group Suite Total tests run: 2, Failures: 0, Skips: 0 ===============================================
As you can see from the previous test results TestNG executed two methods from the group include-group
and excluded the third method that belonged to the group exclude-group
, which was excluded from the test execution.
6. Using regular expressions
While configuring your tests for including or excluding groups, TestNG allows the user to use regular expressions.
This is similar to including and excluding the test methods that we covered earlier. This helps users to include and exclude groups based on a name search.
Let’s learn how to exclude tests based on regex-based name matching.
package com.howtodoinjava.groupExamples; import org.testng.annotations.Test; public class RegularExpressionGroupTest { @Test(groups = { "include-test-one" }) public void testMethodOne() { System.out.println("Test method one"); } @Test(groups = { "include-test-two" }) public void testMethodTwo() { System.out.println("Test method two"); } @Test(groups = { "test-one-exclude" }) public void testMethodThree() { System.out.println("Test method three"); } @Test(groups = { "test-two-exclude" }) public void testMethodFour() { System.out.println("Test method Four"); } }
And testng.xml
file.
<suite name="Regular Exp. Group Suite" verbose="1"> <test name="Regular Exp. Test"> <groups> <run> <include name="include.*" /> <exclude name=".*exclude" /> </run> </groups> <classes> <class name="com.howtodoinjava.groupExamples.RegularExpressionGroupTest" /> </classes> </test> </suite>
The preceding XML contains a simple test in which all the groups with a name starting with include are included, whereas all the groups with name ending with exclude are excluded from your test execution.
Now run the testng.xml
file and you will get below result in console.
Test method one Test method two =============================================== Regular Exp. Group Suite Total tests run: 2, Failures: 0, Skips: 0 ===============================================
Here TestNG executed two methods that belong to groups with a name starting with include and excluded the test methods that belonged to groups with names ending with exclude.
6. Default groups
Sometimes we may need to assign a default group to a set of test methods that belong to a class. This way all the public methods that belong to the said class will automatically become TestNG test methods and become part of the said group.
This can be achieved by using the @Test
annotation at class level and defining the default group in the said @Test
annotation.
@Test(groups={"default-group"}) public class DefaultGroup { public void testMethodOne(){ System.out.println("Test method one."); } public void testMethodTwo(){ System.out.println("Test method two."); } @Test(groups={"test-group"}) public void testMethodThree(){ System.out.println("Test method three."); } }
7. Group of groups or ‘Meta Groups’
TestNG allows users to create groups out of existing groups and then use them during the creation of the test suite. You can create new groups by including and excluding certain groups and then use them.
Let’s create a sample test program and learn how to create a group of groups called MetaGroups.
package com.howtodoinjava.groupExamples; import org.testng.annotations.Test; public class RegularExpressionGroupTest { @Test(groups = { "include-test-one" }) public void testMethodOne() { System.out.println("Test method one"); } @Test(groups = { "include-test-two" }) public void testMethodTwo() { System.out.println("Test method two"); } @Test(groups = { "test-one-exclude" }) public void testMethodThree() { System.out.println("Test method three"); } @Test(groups = { "test-two-exclude" }) public void testMethodFour() { System.out.println("Test method Four"); } }
Now create testng.xml
file like below:
<suite name="Group of group Suite" verbose="1"> <test name="Group of group Test"> <groups> <define name="include-group"> <include name="include-test-one" /> <include name="include-test-two" /> </define> <define name="exclude-group"> <include name="test-one-exclude" /> <include name="test-two-exclude" /> </define> <run> <include name="include-group" /> <exclude name="exclude-group" /> </run> </groups> <classes> <class name="com.howtodoinjava.groupExamples.RegularExpressionGroupTest" /> </classes> </test> </suite>
Here two groups of groups have been defined inside the test, and then these groups are used for test execution. The MetaGroup is created using the <define> tag inside the groups
tag.
The name of the new group is defined using the name
attribute under the define
tag. Groups are included and excluded from the new group by using the include
and exclude
tags.
Now run the testng.xml
test and it will produce below result in console:
Test method one Test method two =============================================== Group of group Suite Total tests run: 2, Failures: 0, Skips: 0 ===============================================
Here, testNG executed only two methods, as mentioned in the included-group
group and excluded the test methods that belong to the excluded-group
. You can define as many groups of groups as you want.
That’s all related to test groups in TestNG. Let me know if you have any queries.
Happy Learning !!
mad
what if i donot give exclude under run tag?
Tamara
i have concern regarding parameters that should get passed while running test case what if we have parameters to get passed how can we handle this ?
Suketa
I found this very useful but under the topic “Assigning Default” the line says “This way all the public methods ” but observed it works not only for public methods but also work for private methods. Thanks
Pankaj Kumar
Hi Lokesh
Wonderful and detailed tutorials best from other i found in net
however i have 2 concerns
1. Some links are not working like
https://www.javadoc.io/doc/org.testng/testng/6.8.21/org/testng/annotations/BeforeSuite.html
https://www.javadoc.io/doc/org.testng/testng/6.8.21/org/testng/annotations/Test.html
2. What about priority to run the script in specific groups
@Test(priority=1,groups= {“registration”})
Suppose i have 10 different script for registration page and i like to run them in some order how to do it ? The above code i found is not working using maven.
With Regards
Pankaj
Suketa
It is working for me. @Test(priority=1,groups= {“registration”})