Writing Integration Tests with Docker Compose and JUnit

An integration test verifies the communication paths and interactions between components to detect interface defects.

— Martin Fowler

In my current project we are building a mobility platform using a microservice architecture. Because all of our services are interacting with external componentes like e.g. DynamoDB, PostgreSQL, Apache Kafka, or etcd we started looking for a simple and efficient way to write integration tests in Java.

After a little research we came up with a minimalistic test setup that makes use of Docker Compose to spin up the dependencies, JUnit and Gradle to execute the tests, and docker-compose-rule to hold it all together.

Here is how you do it…

Let’s get started

In the following tutorial I will show you how to setup and run a simple integration test that verifies the CRUD functionality of a service using DynamoDB. You will then be able to adopt these techniques to write integration tests for your own use cases.

The entire tutorial should take no longer than 20 minutes.

Prerequisites

In order to run the code of this tutorial on your machine you need to install Docker and Docker Compose . That is all.

The entire source code of this tutorial can be found on https://github.com/tomasulo/docker-compose-integration-tests .

Step 1: Setup your build.gradle

The first step is to setup our Gradle build. We do this by adjusting the build.gradle.

We are using docker-compose-rule in our tests to orchestrate the interaction with Docker Compose. For this we need to add their bintray repository to our `repositories` configuration and define a testCompile dependency on 'com.palantir.docker.compose:docker-compose-rule-junit4:0.31.1'.

Then we also want to separate our unit tests from our integration tests so that we can run the Gradle tasks test and integrationTest independently.

We do this by creating a second test task called integrationTest in which we will include a category (more about that later). The regular test task also needs to be adjusted to exclude the aforementioned category.

The complete build.gradle now looks like this:

1plugins {
2    id 'java'
3    id 'idea'
4}
5 
6repositories {
7    mavenCentral()
8    maven {
9    // docker-compose-rule is published on bintray
10    url 'https://dl.bintray.com/palantir/releases'
11    }
12}
13 
14dependencies {
15    compile 'com.amazonaws:aws-java-sdk-dynamodb:1.11.86'
16    compile 'org.apache.commons:commons-lang3:3.5'
17    compile 'ch.qos.logback:logback-classic:1.1.10'
18 
19    testCompile 'junit:junit:4.12'
20    testCompile 'org.assertj:assertj-core:3.6.2'
21    testCompile 'com.palantir.docker.compose:docker-compose-rule-junit4:0.31.1'
22}
23 
24test {
25    useJUnit {
26        excludeCategories 'com.tomasulo.sample.IntegrationTest'
27    }
28}
29 
30task integrationTest(type: Test) {
31    useJUnit {
32        includeCategories 'com.tomasulo.sample.IntegrationTest'
33    }
34}

Step 2: Setup your docker-compose file

The next step is to configure the docker-compose file for the external component we want our tests to interact with. In this tutorial we are spinning up an instance of DynamoDB using the docker image peopleperhour/dynamodb.

The complete configuration looks like this:

1version: "3"
2
3services:
4  dynamodb:
5    image: peopleperhour/dynamodb
6    environment:
7      - "awsRegion=EU_WEST_1"
8      - "awsAccessKey=KEY"
9      - "awsSecretKey=SECRET_KEY"
10    ports:
11      - "8000"
12    hostname: dynamodb

Make sure you are not exposing any ports to avoid conflicts when e.g. running the tests in a CI environment.

In order to make the docker-compose-dynamodb.yml available for our tests we will put it into the folder src/test/resources. You can then test your configuration by starting the container with docker-compose -f docker-compose-dynamodb.yml up and tearing it back down with docker-compose -f docker-compose-dynamodb-yml down.

You can find more information about the docker-compose file here: https://docs.docker.com/compose/compose-file/ .

Step 3: Implementation

Now we are going to configure our tests. But before we do that you should create the category interface we previously talked about:

1public interface IntegrationTest {
2}

We can then annotate the integration tests with that category:

1@Category(IntegrationTest.class)
2public class UserRepositoryIntegrationTest {

The next step is to configure the docker-compose-rule for our purpose. We are using a @ClassRule for this:

1@ClassRule
2public static DockerComposeRule docker = DockerComposeRule.builder()
3    .file("src/test/resources/docker-compose-dynamodb.yml")
4    .waitingForService(DYNAMODB, HealthChecks.toHaveAllPortsOpen())
5    .build();

This configuration will make sure that the service specified in the docker-compose file will be started before the tests and also torn down after.

More info about how to configure the docker-compose-rule can be found here: https://github.com/palantir/docker-compose-rule .

We can now use DockerComposeRule docker in our @BeforeClass to get the external port assigned from Docker to configure the DynamoDB connection:

1private static UserRepository repository;
2 
3@BeforeClass
4public static void initialize() {
5    DockerPort dynamodb = docker.containers()
6        .container(DYNAMODB)
7        .port(DATABASE_PORT);
8    String dynamoEndpoint = String.format("http://%s:%s", dynamodb.getIp(), dynamodb.getExternalPort());
9    repository = new UserRepository(dynamoEndpoint,
10        "KEY",
11        "SECRET_KEY",
12        "EU_WEST_1");
13}

That is all the configuration we need. You are now ready to implement tests and business logic.

Step 4: Profit

Then you are done. It is really that simple.

You can now run ./gradlew clean integrationTest to verify your code in seconds. Or you could simply use the regular JUnit runner of your IDE (e.g. IntelliJ) and even debug your code.

Testing integration points with external components this way really helped us to gain further confidence in our code and also allowed us to test new functionality quickly and effectively.

I hope this tutorial enables you to do the same!

P.S.:
I highly recommend reading https://martinfowler.com/articles/microservice-testing/ if you are interested in more test-patterns for microservices!