Sharing test code
It’s common practice to build some kind of extensible, customizable application that can be deployed with a variety of customizations for different clients. When the base set of functionality is covered by integration and end-to-end tests, it is desirable to run these common tests on the different customized versions of the product.
This was the challenge we faced with my colleagues on a recent project, where we developed a Java based web application that was intended to be deployed with different configurations and customized look and feel. The base version of the application was functional on it’s own, implemented in a Maven war project, had some sensible defaults and the core functionality. The client specific versions, also Maven war projects, provided custom design, configuration and additional functionality. The base project already had quite a lot of Cucumber feature specification that drove end-to-end tests to verify the application through the browser via the WebDriver API. We intended to find an elegant way to run this test suite on the client specific projects, and extend it to cover the additional functionality.
The Really Extensible Calculator
For the sake of simplicity, here is a reduced similar problem that lacks the complexity of a web application, but hopefully able to demonstrate the need for shared integration tests. The sample project is available on Github.
Suppose we have Calculator that can be configured to support many different operations. It has two public methods:
- performOperation, that takes a string as the name of the operation to be performed and two integers as it’s operands
- getResult, that returns the result of the last operation
The Calculator reads a configuration file at startup to determine which operations to support, and what implementation do the operations have.
Implementations of the Operation interface must provide the perform method, for example, our basic feature set might contain an addition:
To register an Operation in the Calculator, one must declare it in the config.properties:
To extend this system, in the case of Maven, we simply need to declare the Calculator as a dependency, provide additional Operations and our custom configuration file.
Integration tests for the base functionality
The Calculator project is documented and tested by Cucumber feature specifications and some glue code. The feature files are run with the JUnit Cucumber runner, as defined below.
The feature directory contains the Cucumber feature files in all projects. It’s referenced via the classpath, not with direct file path. It is necessary to ensure that the test runner takes all feature files into account, even if they are in a dependency in some jar file.
The specification for the base Addition feature is the following:
To make this specification living, we have to provide the appropriate glue code. The first and the last sentence in this scenario is pretty general, not specific to the addition feature. In order to make the glue code more extensible we implement the general step definitions in a separate class.
And the glue code for the addition operation:
Sharing test code and feature files
At this point we can easily test and extend our project with the built-in facilities of Maven. For example, we might create a new project that simply depend on the core module, and extend it with a Multiplication operation. To do that, we just have to provide one implementation class, and a modified configuration.
To run the test suite in the extended project, it must depend on the required test dependencies. One way to do that with Maven is to provide a common parent project that declares the necessary test dependencies.
There are a few things to be done to run the integration test suite on the new project. First, the core module have to export it’s test code and feature files. We can create a test-jar artifact with the maven-jar-plugin.
The exported test-jar artifact contains the compiled tests for the project and the test resources of the project.
The second step is to use these tests in the new project. There are two things to it. In it’s test scope the extended project has to depend on the test-jar artifact of the core project (as well as to the main artifact on it’s compile scope of course).
Then we have to tell Maven to run these included tests. From the maven-surefire-plugin version 2.15 there is a way to scan dependencies for tests to run. (Note: it works the same way with the maven-failsafe-plugin.)
This way the core module’s test classes and feature files are part of the test suite of the extended project.
We might extend this test suite easily, we only have to provide the feature file and the step definitions for the new functionality:
If we build the extended project, all the tests for the core module and the new functionality should run.
I think attached tests are the most suitable to share integration tests, but in some cases it might be beneficial for sharing tests with smaller scope, such as unit tests to catch some weird bugs. For example if someone creates a class with a FQN that exist in a dependency might violate some of it’s invariants. In this case these tests demonstrates that their subject works fine in integration with it’s current environment.