Split unit and integration tests

8 minute read

Introduction

The aim of this article is not to impose a solution, but to show different ways and tools to split unit and integration tests. We all encountered the need to write tests in our developments, the interest of each kind of tests is easy to found in several web sources or others (for example see http://softwaretestingfundamentals.com/unit-testing/), then i will not develop this part here. We will see below, how to split unit and integration tests with several methods of 2 dependency managers (Maven and Gradle), concerning Java source code. In conclusion you will find a succinct comparison of all the methods.


Why ?

magic

Have you already read a lot of articles available on the Web, that talk about unit and integration tests, the aim of each one and the necessity of split them properly. You should also know the following figure (or a similar one), which shows what the code coverage by each type of test should be theoretically:

pyramid

So, I will not waste time on explain general reasons to make it well. Instead, I will enumerate why, from developer’s point of view, that could be great:

  • organization of code: in order to not waste time during development or debugging, it’s necessary to identify and distinguish unit from integration tests.
  • dependencies: by definition, unit tests should need no frameworks to be ran (jdk and JUnit should be enough), so the number of dependencies is very low compared to integration tests. This is linked directly to the next point.
  • compilation time: integration tests often need all frameworks necessary at runtime. That increases significantly the compilation time. In addition, you often need some external tools (Database, external API…) So, splitting unit from integration tests allows you to keep compilation short and run your tests as fast as possible.

How ?

The most common dependency manager for Java is Maven. Its a common way to build our projects in our team. Another one is Gradle that allows more flexibility. Take the following project:

project structure

This web application exposes a http endpoint and depends of a dao module that read data from Database. A very standard use case for the most applications.


Maven

Tools

In order to build a project, Maven is based on plugins. They are launched during many phases of the life cycle. Here the plugins that we will customize to allow the expected behavior:

  • compiler: to configure the proper source code input/output directories
  • resources: to copy resources from a directory to specific directory in target directory (or another)
  • surefire: to launch the unit tests and to generate execution report
  • failsafe: to launch the integration tests and to generate execution report

Split Methods

Naming Convention

Based on naming convention, the applicative source code is put in src/main/java directory and test code in src/test/java directory.

package structure

You could use the default configuration of Surefire and Failsafe plugins to target tests.

<includes>
    <include>**/Test*.class</include>
    <include>**/*Test.class</include>
    <include>**/*Tests.class</include>
    <include>**/*TestCase.class</include>
</includes>
<includes>
    <include>**/IT*.class</include>
    <include>**/*IT.class</include>
    <include>**/*ITCase.class</include>
</includes>

The tests are launched by the following commands:


# unit tests
mvn test

# integration tests
mvn verify

Advantages
  • simple
  • no plugins configuration is needed
  • based on Maven lifecycle :
    • ut+it compilation : test-compile phase
    • ut running : test phase
    • it running : verify phase
Disadvantages
  • Maven compiler plugin is not able to distinguish unit and integration tests, because they have the same root directory src/test/java, so they are built together and the compilation time is the maximum that you could have for the project.
  • no separated directories configured, that can lead to difficulties to develop or to debug.

Dedicated packages

Starting with the previous structure project, the way to split the Unit Tests (ut) and Integration Tests (it) is to create a specific and dedicated package. For example it package, for integration tests and ut for unit tests.

package test

If you want to avoid previous undesirable effects, you have to configure some plugins. Such as following in order to separate directories:

<includes>
    <include>**/ut/**/*.class</include>
</includes>
<excludes>
    <exclude>**/it/**/*.class</exclude>
</excludes>
<includes>
        <include>**/it/**/*.class</include>
</includes>
<excludes>
    <exclude>**/ut/**/*.class</exclude>
</excludes>

The tests are launched by the following commands:

# unit tests
mvn test

# integration tests
mvn verify
Advantages

The same as Naming Convention but:

  • on the right way to split physically ut and it tests
  • all name for the classes are allowed
Disadvantages

The same as Naming Convention concerning the compilation time.


Junit Category

To use the Junit Category mechanism, we have to add 2 interfaces:

public interface UnitaryTest {
}

public interface IntegrationTest {
}

For the example, this interfaces are put in a additional module of the project.

category

All that we have to do is put an Junit Annotation on the test class

@Category(value = IntegrationTest.class)

or

@Category(value = UnitaryTest.class)

We’ll put a wildcard filter on the test classes and specify the category for each type of tests:

<includes>
    <include>**/*.class</include>    
</includes>
 <groups>my.incredible.package.of.test.categories.UnitaryTest</groups>
<includes>
        <include>**/*.class</include>
</includes>
 <groups>my.incredible.package.of.test.categories.IntegrationTest</groups>

The tests are launched by the following commands:

# unit tests
mvn test

# integration tests
mvn verify
Advantages

The same as Naming Convention but:

  • easily identifying ut and it tests using annotations
Disadvantages

The same as Naming Convention but :

  • developers can easily forget to put the annotation, and the test classes will be ignored.

Dedicated Module

In this method, we put all the it tests (and theirs resources) on a specific module of the project, the ut tests are kept in each module into the src/test directory.

dedicated module

Nothing more, but to configure the surefire plugin only :

<includes>
    <include>**/*.class</include>
</includes>

The tests are launched by the following commands:

# unit tests only
mvn -pl !integration-test clean test
# or
mvn -pl test-dao,test-core,test-web clean test

# integration test only
mvn -pl integration-test clean verify

# all
mvn clean verify

Advantages
  • no naming convention
  • proper method to split ut and it tests in different places
  • compilation time reduced for it and ut tests.
Disadvantages
  • not based on Maven lifecycle
  • developer must know the maven command line option to skip targeted tests
  • take away the it tests code from the applicative code
  • remove the possibility to test applicative modules distinctly (for the it tests)
  • compilation time increased for it tests, because of grouping of all module tests.

Dedicated source directory

In this method, all the modules have to follow this structure:

directory

  1. src/main: contains all the applicative classes and resources
  2. src/test: contains all the unit test classes and resources
  3. src/integration-test: contains all the integration test classes and all the integration resources.

This project structure need a special configuration of the different modules:

# Compile the unit test classes to the unit test output directory
<execution>
    <id>default-testCompile</id>
    <goals>
        <goal>testCompile</goal>
    </goals>
    <phase>test-compile</phase>
    <configuration>        
        <outputDirectory>${basedir}/target/test-classes</outputDirectory>
        <compileSourceRoots>
            <sourceRoot>${basedir}/src/test/java</sourceRoot>
        </compileSourceRoots>
    </configuration>
</execution>
# Compile the integration test classes to the integration test output directory
<execution>
    <id>compile-integration-tests</id>
    <goals>
        <goal>testCompile</goal>
    </goals>
    <phase>pre-integration-test</phase>
    <configuration>
        <outputDirectory>${basedir}/target/integration-test-classes</outputDirectory>
        <compileSourceRoots>
            <sourceRoot>${basedir}/src/integration-test/java</sourceRoot>
        </compileSourceRoots>
    </configuration>
</execution>
# Copy resources for the unit test resources files to the unit test output directory
<execution>
    <id>default-testResources</id>
    <goals>
        <goal>testResources</goal>
    </goals>
    <phase>process-test-resources</phase>
    <configuration>
        <outputDirectory>${basedir}/target/test-classes</outputDirectory>
        <resources>
            <resource>
                <directory>${basedir}/src/test/resources</directory>
            </resource>
        </resources>
    </configuration>
</execution>
# Copy resources for the integration test resources files to the integration test output directory 
<execution>
    <id>add-integration-test-resources</id>
    <goals>
        <goal>copy-resources</goal>
    </goals>
    <phase>pre-integration-test</phase>
    <configuration>
        <outputDirectory>${basedir}/target/integration-test-classes</outputDirectory>
        <resources>
            <resource>
                <directory>${basedir}/src/integration-test/resources</directory>
            </resource>
        </resources>
    </configuration>
</execution>
<configuration>
  <testClassesDirectory>${basedir}/target/test-classes</testClassesDirectory>
  <testSourceDirectory>${basedir}/src/test/java</testSourceDirectory>
  <includes>
    <include>**/*.class</include>
  </includes>
  <argLine>${jacoco.agent.ut.arg}</argLine>
  <testFailureIgnore>false</testFailureIgnore>
</configuration>
<executions>
  <execution>
    <id>integration-test</id>
    <phase>integration-test</phase>
    <goals>
      <goal>integration-test</goal>
    </goals>
    <configuration>
      <includes>
        <include>**/*.class</include>
      </includes>
      <classesDirectory>${project.build.outputDirectory}</classesDirectory>
      <testClassesDirectory>${basedir}/target/integration-test-classes</testClassesDirectory>
      <testSourceDirectory>${basedir}/src/integration-test/java</testSourceDirectory>
    </configuration>
  </execution>
  <execution>
    <id>verify</id>
    <phase>verify</phase>
    <goals>
      <goal>verify</goal>
    </goals>
    <configuration>
      <testFailureIgnore>false</testFailureIgnore>
    </configuration>
  </execution>
</executions>

The tests are launched by the following commands:

# unit tests
mvn test

# integration tests
mvn verify
Advantages

the sames as Dedicated module but :

  • proper method to split ut and it tests in different places, but near the applicative code
  • based on Maven lifecycle:
    • ut compilation : test-compile phase
    • ut running : test phase
    • it compilation : pre-integration-test phase
    • it running : verify phase
  • services classpath can be respected by test classes
  • keep the possibility to test applicative modules distinctly (it or ut tests)
Disadvantages
  • complex configuration, but it is made once on a project lifecycle
  • developer has to know the maven lifecycle to skip targeted tests

Gradle

Gradle is a build-automation system that has been thought to build many languages, so code splitting is no more by convention. This method is based on the same project structure than see at dedicated-source-directory. All we have to do it’s to configure a specific tasks for integration tests. I’ve based my configuration on this tutorial: multi-project-builds.

# add a plugin to run integration-test
plugins {
    id "com.coditory.integration-test" version "1.0.8"
}

# define the source set for integration test
sourceSets {

    integrationTest {

       java.srcDir 'src/integration-test/java'
       resources.srcDir 'src/integration-test/resources'

        compileClasspath += main.output + test.output
        runtimeClasspath += main.output + test.output
    }
}

# define integration test configurations based on standard test configuration
configurations {
    integrationTestImplementation.extendsFrom testImplementation
    integrationTestRuntimeOnly.extendsFrom testRuntime
}

# define the integration test task
task integrationTest(type: Test) {
    testClassesDirs = sourceSets.integrationTest.output.classesDirs
    classpath = sourceSets.integrationTest.runtimeClasspath
}

# add the integration test task on the check list
check.dependsOn integrationTest

The tests are launched by the following commands:

# unit test 
gradle test
# integration test
gradle integrationtest
Advantages
  • Simple
  • built in
  • the same configuration can be used for different languages.
Disadvantages
  • not the most used tool in our historical projects, but its usage increase.

Conclusion

A short summary about that we saw previously could be put in the following table, , where stars symbolize the score of an item :

  • one star is a low score
  • five stars is a high score
 naming convention (Maven)dedicated packages (Maven)category (Maven)dedicated module (Maven)source directory (Maven)Gradle
complexityfourStarfourStarfourStarfourStarfiveStarfiveStar
configurationfiveStarthreeStarthreeStarfourStarthreeStarfourStar
compiler lifecyclefourStarfourStarfourStaroneStarfiveStarfiveStar
source codetwoStaroneStaroneStarfourStarfiveStarfiveStar

As we saw, unit and integration tests have not the same goal or the same impact on the compilation time. We need to split tests in order to increase readability, maintainability and increase the development experience. This can be implemented through many ways and the choice of one solution must be made with caution. Finally, we saw that Gradle have some advantages over Maven. It gives us more flexibility and simplicity. The same reasons why Google choose it as official builder for Android projects.


Written by

Davy Garçon

Architect, GM. I love development but also board/video/rolePlay games. #java #angular #vue.js