Two ways of skipping tests in Maven

Let me share an interesting story that we faced at work recently. We have a complex application with multiple modules, and we wanted to set up a new dev environment. It's a big project, and we are usually in a hurry (or just lazy), so to save time we tried to check out the latest version of the project and build it without running the test suites.

We passed the -Dmaven.test.skip parameter to the build to the desired effect. But the build failed, complaining about a missing internal test library, found in one of the modules. This gave us a hint to rebuild the project without skipping the tests, and the build succeeded.

It was quite early in the morning and this really got me by surprise. The story seemed really interesting because skipping the tests used to have the opposite effect.

Turns out there are multiple ways of skipping tests with Maven. We normally use skipTests, but maven.test.skip seemed just as fine at the moment. According to the documentation of the maven-surefire-plugin, you can skip the tests with the following flags:

• -Dmaven.test.skip
• -Dmaven.test.skip.exec (deprecated)
• -DskipTests

The first option ignores all test sources altogether. They are not compiled and thus no test artifacts are produced. The last two are synonyms. They just don't run the tests, but all test files are compiled.

This made a difference in our case because one of our modules depends on the test artifact of another.

A module usually export one artifact, typically a jar file assembled from its main classes and resources. However, it's common to build artifacts for the source code and Javadoc as well. (For example, it's a requirement for publishing to Maven Central.)

In addition, you can also generate an artifact from the test classes and resources as well, for example, to run test cases in different modules or to share a piece of test library. For example, see this project where one module exports all its Cucumber test cases for others to consume it.

Here is what the documentation says about the maven.test.skip flag:

Set this to "true" to bypass unit tests entirely. Its use is NOT RECOMMENDED, especially if you enable it using the "maven.test.skip" property, because maven.test.skip disables both running the tests and compiling the tests. Consider using the skipTests parameter instead.

The bottom line is that you shouldn't skip test execution in the first place, but if you have to, use the skipTests flag as it compiles the tests at least, just skipping their execution. Not using maven.test.skip has an additional benefit that you can be sure that at least your test classes can be compiled.