Multiple test executions with maven-surefire-plugin

Hello boys & girls,

After a long period without no technical article, today, the time comes for a new one. It’s time to talk about how it’s possible to have multiple test executions using maven-surefire-plugin. First question is why we would like to multiple executions? Why one execution is not enough?

Let’s say you want to split all your existing tests into two main categories: some tests that are eligible to be executed in parallel (and will be a good idea to do this because will reduce a lot your entire execution time) and some tests that are not eligible to be executed in parallel (due to some constraints on your own API) and its mandatory to be executed sequentially, using just one thread.

How we can reach this approach, using maven-surefire-plugin? As you probably already know, maven-surefire-plugin, together with JUnit and be used to plan and execute all your scenarios. In your POM file, if you have something like this, if you want to execute all scenarios sequentially:

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0-M5</version>
            <executions>
                <execution>
                    <id>run-all-tests</id>
                    <goals>
                        <goal>test</goal>
                    </goals>
                    <configuration>
                        <includes>
                            <include>**/*Test.java</include>
                        </includes>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

To achieve our needs we may change the above implementation with the following one:

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0-M5</version>
            <executions>
                <execution>
                    <id>run-parallel-tests</id>
                    <goals>
                        <goal>test</goal>
                    </goals>
                    <configuration>
                        <includes>
                            <include>**/*ParallelTest.java</include>
                        </includes>
                        <parallel>methods</parallel>
                        <threadCount>4</threadCount>
                        <testFailureIgnore>true</testFailureIgnore>
                    </configuration>
                </execution>
                <execution>
                    <id>run-sequential-tests</id>
                    <goals>
                        <goal>test</goal>
                    </goals>
                    <configuration>
                        <includes>
                            <include>**/*SequentialTest.java</include>
                        </includes>
                        <testFailureIgnore>true</testFailureIgnore>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

All tests suffixed with ParallelTest will be grouped together and executed in parallel using 4 threads. Any kind of failures will be ignored, just to be able to continue with sequential flow. After all parallel tests are completed, sequential tests are planned for execution, but, as you can see, with a different configuration. No parallelization enabled in this case. All failures are ignored again.

That’s it! We managed to implement our desired approach. Happy coding! 🙂