Overview

Docker Registry or How to Run your own Private Docker Image Repository

16 Comments

Docker allows to bundle artifacts and configurations in an image. These images run as light weight system-level virtual machines. In my previous articles, I showed how to use Docker in general  and how to use networking. In this article, I will show you how to manage images in a private Docker registry — basically a git repository for images — and run a production grade registry. As in my previous articles, a fully working example is available on Github to get you started with your own private Docker registry.

Every single command in a Dockerfile yields a new Docker image with an individual id similar to a commit in git. This commit can be tagged for easy reference with a Docker Tag. In addition, tags are the means to share images on public and private repositories.

Tags

When you build a Docker image using a Dockerfile, you can set a tag for the final image by passing the parameter -t <tag>. You can also tag any other image with docker tag <image> <tag>. The expression tag has evolved in Docker quite a lot and its meaning blurred. The syntax for a tag is repository:[tag]. So we have a recursive definition of tag 😉

In general, a repository is a collection of images which are hosted in a registry. When you use the public Docker.io repository — also called the Docker Index —, your repository is <username>/<repo_name> — cf [1].

For example, I shared the registry-demo container to the Docker Index by first tagging and then pushing it:1

> docker build lukaspustina/registry-demo registry
> docker push lukaspustina/registry-demo

In case you want to use a private registry, you need to set the URL to your private registry as the username. First tag it accordingly and then push it — assuming your registry is listing on localhost port 5000:

> docker tag lukaspustina/registry-demo localhost:5000/registry-demo
> docker push localhost:5000/registry-demo

Registry

You can download the docker registry from here. The configuration of a registry is straight forward and described in the corresponding Readme. You can define different modes a registry should run called flavors, e.g. for debugging, different storage backend etc. Docker registry supports multiple storage backend. You can choose from local file system, Amazon S3, OpenStack Glance etc.

If you run a production registry with multiple workers, it is crucial to set a “secret_key” in the configuration file config/config.yml to a 64 characters long value. Otherwise, you will get very confusing errors if you run more than one worker thread.

In order to retrieve an image from a repository, run docker pull <name> where name denotes the repo_name given above. If you want to select a specifically tagged version, you can supply the tag with the -t option.

For an image from the Docker Index, run
> docker pull lukaspustina/registry-demo

and for a private registry, run
> docker pull localhost:5000/registry-demo

Demo

You can find my demo project on GitHub. You can either run it directly on a Docker compatible OS like Linux and OS X or use Vagrantfile to setup a Virtual Box — see the Readme on GitHib for details.

The demo consists of two steps. The first step builds a production-ready Docker Image containing the Docker registry and two Python images for 2.7 and 3.3 which are pushed to the registry.2 For this demo, the two Python version are used as two versions of the same artifact ‘python’ to demonstrate how to use tags for versioning images of the same repository.
> make build

You can see how the docker layers are uploaded to the registry:

Pushing repository localhost:5000/python (2 tags)
Image 511136ea3c5a Image successfully pushed
Image cc7385a89304 Image successfully pushed
Image 24ba2ee5d982 Image successfully pushed
Image a49658c12a4f Image successfully pushed
Image 7adcddf9f908 Image successfully pushed
Image 818c514e00c0 Image successfully pushed
Image 480cb1b9f62c Image successfully pushed
Image 9eeddd8616b7 Image successfully pushed
Image c26adcc0bc2a Image successfully pushed
Pushing tag for rev [c26adcc0bc2a] on
  {http://localhost:5000/v1/repositories/python/tags/python-2}
Image c04e8b52daba Image successfully pushed
Image 1ebdf4e79ef1 Image successfully pushed
Pushing tag for rev [1ebdf4e79ef1] on
  {http://localhost:5000/v1/repositories/python/tags/python-3}

You can also see that tags for the two Python version are also transmitted allowing a user to pull only a specific version.

The second step pulls the Python images and run them demonstrating the whole life cycle of build, pushing, pulling, and running image with your own, private Docker registry.
> make start-registry pull run-python-2 run-python-3 stop-registry

You can see how the individual layers of the Python images are gathered and afterwards run:

docker pull localhost:5000/python
Pulling repository localhost:5000/python
1ebdf4e79ef1: Download complete
c26adcc0bc2a: Download complete
511136ea3c5a: Download complete
cc7385a89304: Download complete
24ba2ee5d982: Download complete
a49658c12a4f: Download complete
7adcddf9f908: Download complete
818c514e00c0: Download complete
480cb1b9f62c: Download complete
c04e8b52daba: Download complete
9eeddd8616b7: Download complete
...
docker run localhost:5000/python:python-2
Python version is 2.7.5+
docker run localhost:5000/python:python-3
Python version is 3.3.2+

Running a Production Registry

At CenterDevice we are running a Docker registry on our Jenkins CI server. The main command line to run the registry is
> SETTINGS_FLAVOR=prod DOCKER_REGISTRY_CONFIG=/etc/docker/config.yml gunicorn -k gevent -b localhost:5000 --max-requests 100 --graceful-timeout 3600 -t 3600 -w 8 wsgi:application

which sets the flavor and the location of the configuration file. In addition, it starts 8 worker threads and listens only on localhost. Since the current version of Docker registry has no authentication, we use ssh tunnels for this purpose. The registry instance only listens on localhost. If you want to connect to the registry, you need to create a tunnel first with
> ssh -N -L 5000:localhost:5000 user@server

This requires a valid user on our server and enforcing authentication and authorization. For the storage backend we use S3.

The Dockerfile  in the GitHub demo project is almost the same as the one we use on our Jenkins. Please feel free adapt it to your needs. The main difference is that the demo uses local file storage, but the config.yml template file is already prepared for S3. See the Readme for details. If you’re feeling lazy, you also find this Docker image on Docker Index.

There’s more

At CenterDevice, we like to use private Docker registries because they allow us to safely share Docker images in our organization. We maintain all our backend service as well as our app images in a private registry. In this way, a developer only needs to pull changed images to update his development environment. In the future, we plan to use these images for production environments, too.

Docker is evolving very quickly, with a minor version update planed each month. So stay tuned for more articles about how we use Docker at CenterDevice (German readers can read more about it here).

Footnotes

1. You can run my shared registry directly: > docker run --name registry -d -p 5000:5000 -v <absolute local path>:/docker-registry-storage lukaspustina/registry-demo.
2. Please be patient as the build process takes quite some time.

Tags

Kommentare

  • Karthi Kulandaivelu

    28. March 2014 von Karthi Kulandaivelu

    How do I find out the list of images or containers stored in my local registry? Is there something equivalent to “docker images” ?

    I am probably missing something obvious, but as of now, I am unable to find any usefull suggestions. Thanks

  • Murphy Randle

    30. April 2014 von Murphy Randle

    This article was super helpful in figuring out what kind of auth to use, etc. Thanks for taking the time to write it!

  • David

    Nice Info Shared!I would like to add:
    Use of the registry, without the index, which is under the full control of Docker, is best suited for storing images on private networks. The registry spins up in a special mode which restricts communication with the Docker index. All security and authentication needs to be taken care of by the user.

  • Tom

    You can search the registry just fine,

    curl http://localhost:5000/v1/search

    (produces JSON output)

  • gavin

    Is ‘registry’ or ‘repository’ a term that means ‘hosting’? I’m looking for a place to host containers.

  • Peter

    Hi there,

    I am wondering, what you guys do manage the registry. Say – how do you delete an image?

    thanks

    Peter

    • Lukas Pustina

      Hi Peter,
      right now, all we do, is to delete images we don’t need anymore in the S3 bucket. It’s not optimal, but more professional registries are becoming available that do support management.

  • Scott Herzinger

    22. January 2015 von Scott Herzinger

    Hi,

    Your post is very useful. I’m currently working through your demo project. I’ve run into a few speed bumps that appear mostly due to things that have changed in docker since you posted. One thing that I haven’t gotten past is the following command in registry/Dockerfile.

    RUN cd /docker-registry && pip install -r requirements.txt

    I can’t find requirements.txt anywhere in the current repo at https://github.com/lukaspustina/docker-registry-demo and it doesn’t appear to be generated at build time.

    Have you seen this? Suggestions?

    Thanks
    Scott

    • Lukas Pustina

      Hello Scot,

      sorry, for my late reply. I had the flu and didn’t find time to respond earlier.

      You’re right. A lot changed since I wrote the article and the way to use the registry changed. While I had to write a Dockerfile by my self, today’s registry already has one that works perfectly fine and settings only need to be adapted in a derived Dockerfile.

      I updated the GitHub project. Please have a look if it works for you now.

      Cheers,
      Lukas

  • Bogdan Kulbida

    11. April 2015 von Bogdan Kulbida

    Awesome post!
    Please correct the typo…
    “… URL to your private registry as the username. First tag it accordingly and the>>N<< push it — assuming…"

  • sandilya miduthuri

    22. June 2015 von sandilya miduthuri

    How to connect from a client other than local host ?

    • Lukas Pustina

      Hi Sandilya,
      please see section “Running a Production Registry” for how we connect to our registry. Does that answer your question? If not, can you rephrase and be a bit more specific?

      Cheers,
      Lukas

Comment

Your email address will not be published. Required fields are marked *