How to Stop Suite Execution after First Failure in TestNG?

TestNG is the popular Java testing framework that offers annotations, configurations, and features in regards to test execution management. While working with TestNG there are a few cases when you might need to stop the execution of your test suite on the first failed test not to run some additional tests that rely on the successful execution of the earlier tests. The default behavior of the TestNG framework does not provide such an option, but using custom configurations you might do it.

Approach to Stop Suite Execution after First Failure in TestNG

Following are the steps to Stop Suite Execution after First Failure in TestNG.

• Create a Test Class with Multiple Test Methods: To demonstrate stopping test execution after the first failure, we'll need a test class that contains multiple test methods. We will set up one method that will fail and others that should not execute once the failure occurs.
• Configure TestNG for Stop on First Failure: There is no direct option available in the configuration file, testng.xml, with TestNG to halt the execution on the first failure. Instead, we can achieve that by the help of a TestListener that listens for the test failures and controls the halting of suite execution programmatically.
• Run the Test Suite: We'll configure TestNG to run our test class using the listener and show how the run would stop once the first failure has occurred.

Example to Stop Suite Execution after First Failure in TestNG

1. Create a Test Class

import org.testng.Assert;
import org.testng.annotations.Test;

public class TestClass {

    @Test public void testMethod1()
    {
        System.out.println("Test Method 1 executed");
        Assert.assertTrue(true); // Passes
    }

    @Test public void testMethod2()
    {
        System.out.println("Test Method 2 executed");
        Assert.fail(
            "Failing test method 2"); // Intentionally fails
    }

    @Test public void testMethod3()
    {
        System.out.println("Test Method 3 executed");
        Assert.assertTrue(true); // Should not execute if we
                                 // stop on failure
    }
}

2. Create a Listener to Stop Execution on Failure

We can use the ITestListener interface from TestNG to create a listener that stops the execution once a test fails.

import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestResult;

public class StopExecutionListener implements ITestListener {

    @Override
    public void onTestFailure(ITestResult result) {
        System.out.println("Test failed: " + result.getName());
        // Stop the test suite on first failure
        throw new RuntimeException("Stopping the suite execution due to test failure");
    }

    @Override
    public void onTestStart(ITestResult result) { }
    @Override
    public void onTestSuccess(ITestResult result) { }
    @Override
    public void onTestSkipped(ITestResult result) { }
    @Override
    public void onTestFailedButWithinSuccessPercentage(ITestResult result) { }
    @Override
    public void onStart(ITestContext context) { }
    @Override
    public void onFinish(ITestContext context) { }
}

3. Configure TestNG to Use the Listener

In your testng.xml file, add the StopExecutionListener to the listener section.

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd" >
<suite name="Suite" verbose="1">
    <listeners>
        <listener class-name="StopExecutionListener" />
    </listeners>
    <test name="TestSuite">
        <classes>
            <class name="TestClass"/>
        </classes>
    </test>
</suite>

4. Run the Suite

When you run the test suite using the TestNG runner, you'll observe that:

• testMethod1 runs successfully.
• testMethod2 fails intentionally.
• The suite stops execution and does not run testMethod3.

Output

Conclusion

A custom ITestListener can be created to stop the execution of a TestNG suite after the first test failure. This is really beneficial where tests depend on successful completions of previous tests; or simply no value is found in running further tests after a critical failure. It, therefore, ensures better test control and reduced runs of unnecessary tests in scenarios where failures are critical.