Beliebte Suchanfragen

Cloud Native

DevOps

IT-Security

Agile Methoden

Java

//

Continuous Integration with Drone on AWS

14.2.2018 | 6 minutes of reading time

Introduction

Drone is a Continuous Delivery system built on container technology and written in Go. It uses a simple configuration yaml, with syntax similar to a docker-compose file, to define and execute pipelines inside Docker containers.

In this article, I show how to set up a Drone build server, running on an AWS EC2 instance, which connects to your (private) GitHub repository and builds and tests your code after each commit.

Prerequisites

You need the following for the example setup:

  • A basic understanding of continuous integration
  • Basic knowledge of AWS as well as an AWS account
  • Basic knowledge of docker
  • A github account
  • A golang project (“Hello World” will do), alternatively use the example Java .drone.yml at the end of this article in conjunction with a simple Maven/Java project

Why Drone?

Let’s take a look at the following .drone.yml file, capable of building a simple Go project:

1pipeline:
2  build:
3    image: golang
4  commands:
5    - go build
6    - go test

That’s it already. Intriguing, isn’t it? Of course there aren’t any notifications for when the build fails at this stage. But this file allows you to get started. Yet, how does it work? With Drone you basically put everything you need to build your software in a Docker container and execute the commands to do so. In the case above, we just need the golang binaries to build our go application; so the standard golang docker image is sufficient. The project is then built and tested with go commands. If any of the commands returns with an exit code > 0, the build fails. Drone took a good look at Travis config file syntax but unlike Travis, Drone can be installed on any system of your choice and doesn’t have to be exclusively used as software as a service. The Drone Team is also planning a fee-based concept, but as it is an OpenSource project under Apache licence, you are free to set it up in your infrastructure.

Let’s begin setting up the whole system.

Setup EC2 instance

We launch an EC2 instance with the default AWS Linux image into a public subnet with a security group open for the following tcp ports: 22, 80, 443, 9418 and 8000. Next, we ssh in, and continue with the Docker installation which is described here: AWS docker on EC2 documentation.

Afterwards, we continue the installation process with docker-compose.

Install docker-compose

still in ssh:

1sudo su
2curl -L "https://github.com/docker/compose/releases/download/1.18.0/docker-compose-$(uname -s)-$(uname -m)" > /usr/local/bin/docker-compose
3chmod +x /usr/local/bin/docker-compose
4exit
5docker-compose —version

Sometimes it is necessary to logout and get back to the EC2 instance to verify the installation. Now we are set AWS-wise, let’s continue with setting up the repository.

The build file for your project

Our example project is a golang project – pretty easy to build, as you have seen in the introduction. Just add a .drone.yml file into the root directory of your go project with the following content:

1pipeline:
2  build:
3    image: golang
4    commands:
5      - go build
6      - go test

The pipeline will fail if any of the commands returns a nonzero exit status, indicating failure. The next step brings repository and pipeline together.

GitHub OAuth consumer

We need to create a new OAuth consumer with a callback URL, to enable Drone to connect to our repositories. In your GitHub account, go to Settings —> Developer settings —> OAuth Apps —> Register
Fill in the form with sensible information and provide the following callback url:

http://[public-ip-of-ec2-instance]:[drone-port]/authorize

as website URL you can enter the following:

http://[public-ip-of-ec2-instance]:[drone-port]

In our case the drone-port is the exposed port from the docker-compose file, which will be 8000. Adapt the AWS security group for the EC2 instance to allow custom tcp port 8000 and http. It should look like this:

Now we need to provide a docker-compose file on the EC2 instance, which will start the Drone server and the Drone agent. On the EC2 instance terminal session, create a docker-compose.yml file with the following content and replace values in the square brackets:

1version: '3'
2
3services:
4  drone-server:
5    image: drone/drone:0.8.1
6    ports:
7      - 8000:8000
8      - 9000:9000
9    volumes:
10      - ./drone:/var/lib/drone/
11    restart: always
12    environment:
13      - DRONE_OPEN=true
14      - DOCKER_API_VERSION=1.26
15      - DRONE_HOST=127.0.0.0
16      - DRONE_SECRET=[RANDOM_SECRET_OF_YOUR_CHOOSING]
17      - DRONE_ADMIN=[YOUR_GITHUB_ACCOUNT]
18      - DRONE_GITHUB=true
19      - DRONE_GITHUB_CLIENT=[CLIENT_KEY_FROM_OAUTH]
20      - DRONE_GITHUB_SECRET=[SECRET_FROM_OAUTH]
21
22  drone-agent:
23    image: drone/agent:0.8.1
24    restart: always
25    depends_on:
26      - drone-server
27    volumes:
28      - /var/run/docker.sock:/var/run/docker.sock
29    environment:
30      - DOCKER_API_VERSION=1.26
31      - DRONE_SERVER=drone-server:9000
32      - DRONE_SECRET=[MUST_BE_SAME_AS_IN_SERVER]

Complete DRONE_GITHUB_CLIENT and DRONE_GITHUB_SECRET with values of the GitHub OAuth application you created. You can use an arbitrary value as DRONE_SECRET, e.g. generate one with the following command:

openssl rand -base64 32

Make sure that the secret is the same for the agent and the server. After that, run the stack with:

docker-compose up

Wait till Drone and the agent have started and can communicate with each other. Relevant log output should be visible in the EC2 terminal. Afterwards visit your EC2 instance at:

http://[public-ip-of-ec2-instance]:8000

Allow access to your account; you can now activate your repository. The only prerequisite for activation is a valid .drone.yml within the root directory of the project. Upon activation, Drone automatically adds webhooks, which will trigger a build every time code is committed to your repository. Manual configuration is not required.

Now you can commit into your repository to trigger a build and monitor the progress in your browser.

Dry run

An additional “nice to have” feature is that you can also „dry“ build your software with the Drone command line tools. Installation e.g. with homebrew:

1brew tap drone/drone
2brew install drone

The following command executed in the directory with your .drone.yml file will run the build locally, allowing you to check before submitting code to your source control repository:

1drone exec

What about Java and other languages?

What do you need to build your Java project? Most of the time you just need Maven, and the „internet“. So here’s the .drone.yml file:

1pipeline:
2  build:
3    image: maven:3.5-alpine
4    commands:
5      - mvn clean package

I think this is pretty neat. What’s more, you can provide any Docker image you want, either from your own registry or from Docker hub, and use the specific tag in your .drone.yml.

Of course if you want to deploy in addition to build, things get a bit more complicated, as for secrets and configuration and so on, but that’s another story.
Furthermore, having a Drone EC2 instance running unprotected in public might not be a good idea. For a more secure approach you could, for instance, configure Drone behind a nginx proxy.

Outlook

To expand your build pipeline and offer continous deployment, you can customize Drone with a variety of plugins to integrate further functionalities into your pipeline. There already are plugins for the AWS build stack, Docker, Kubernetes as well as notification plugins for slack and google mail, just to name a few. In addition you can write your own plugins in bash, go, node and python.

Conclusion

Drone has quite an elegant approach to building your software – the simplicity of the build file struck me straightaway. Although still in beta, you should definitely keep an eye on Drone in the future.

share post

Likes

0

//

More articles in this subject area

Discover exciting further topics and let the codecentric world inspire you.

//

Gemeinsam bessere Projekte umsetzen.

Wir helfen deinem Unternehmen.

Du stehst vor einer großen IT-Herausforderung? Wir sorgen für eine maßgeschneiderte Unterstützung. Informiere dich jetzt.

Hilf uns, noch besser zu werden.

Wir sind immer auf der Suche nach neuen Talenten. Auch für dich ist die passende Stelle dabei.