Migrating your Gitlab Infrastructure into Docker

Bring your GitLab down to see how quickly you can annoy your developers.

If you did that already, you know how sensitive your built environment is, and you should do everything you can to make it healthy and stable. Unfortunately, that sometimes means that you might think twice before deciding to upgrade your GitLab configuration or before upgrading your CI server.

However, when using Docker, those troubles go away.

Today, we will show you how to migrate your current GitLab infrastructure into Docker.

Docker 101

If you have been around in the last couple of years, you probably already heard of Docker. Docker is a Linux Container management toolset.

Linux Containers provide resource limitation and prioritization (CPU, memory, block I/O, network) and application isolation of the execution environment (filesystem, process trees, networking, user IDs). This has obvious benefits in terms of security, but an equally important one is making the management of your CI infrastructure much more enjoyable and less painful.

Docker containers are not persistent because they are based on immutable images. If you restart a container, you’ll restart all the processes also. All changes made in previous executions will be lost because the container is created from the same original image.

To persist data, you need to separate the state (configuration files you want to be able to edit frequently, databases, logs, etc.) from the rest (system and static application files).

The static parts should live inside the docker container, and the state should be moved out to host volume folders that you can mount into your containers. This way, your state will be safe across multiple executions of the container.

Creating the initial state

Assuming that you already installed Docker on your server, the first step is selecting a Docker image that suits our needs. It’s always best to choose an official image as those are well structured and tend to be more up-to-date.

For this post, we selected the Gitlab Community Edition Docker image, which is based on the Omnibus package.

Before running the container we need to create the host volumes and set up the default configuration files that we’ll need in the first execution. There are three host volume folders that we will use:

To set our default configuration first create Gitlab’s configuration file /docker/GitLab/GitLab.rb. Let’s start with something simple:

# Change the external_url to the address your users will type in their browser
external_url 'https://gitlab.yourcompany.com'
nginx['listen_port'] = 443
nginx['listen_https'] = true
# SMTP settings
gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "smtp.yourcompany.com"
gitlab_rails['smtp_port'] = 587
gitlab_rails['smtp_user_name'] = "gitlab"
gitlab_rails['smtp_password'] = "supersecurepassword"
gitlab_rails['smtp_domain'] = "yourcompany.com"
gitlab_rails['smtp_authentication'] = "login"
gitlab_rails['smtp_enable_starttls_auto'] = true
gitlab_rails['gitlab_email_from'] = '[email protected]'

For security purposes, we should enable SSL by default. This GitLab image will look for the SSL certificate in the /ssl/gitlab.yourcompany.com.crt.

Generate your self-signed certificate or even better, purchase a certificate for your GitLab sub-domain and install the files in:

/docker/gitlab/ssl/gitlab.yourdomain.com.crt
/docker/gitlab/ssl/gitlab.yourdomain.com.key

Don’t forget to do a chmod 600 /docker/gitlab/ssl/gitlab.yourdomain.com.*. The easiest way to break your crypto is to give your key away!

Creating the container

We don’t need to create a Dockerfile to set up our container, but it’s best if we do!

We can later tweak things to our taste and it’s easier to remember exactly what settings we used and a convenient starting point to upgrade our configuration. If you are migrating your current non-Docker installation of Gitlab into Docker you’ll need to install the same version.

Don’t worry, you can upgrade later. To specify a version you need to add an existing tag to the image name. In our case, we are using 8.5.7-ce.0.

We’ll use the supervisor to wrap up our service. The supervisor is basically a daemon wrapper that can automatically restart your services if for some reason they become down. This is the look of our Dockerfile:

FROM gitlab / gitlab - ce: 8.5.7 - ce.0
MAINTAINER you@ yourdomain.com

## Setup##
RUN apt - get update &&
    apt - get install - y supervisor &&
    mkdir /
    var / log / supervisor
COPY. / supervisor.conf / etc / supervisor / conf.d /

    EXPOSE 22 443
VOLUME['/etc/gitlab', '/var/log/gitlab', '/var/opt/gitlab']
CMD["/usr/bin/supervisord"]

The FROM instruction specifies which image we are using as a starting ground to build our own. RUN instructions indicate commands that will be executed on the container filesystem when building the image.

You can have multiple RUN instructions or a single one separating commands with &&. It’s best to merge into a single one as each instruction will result in a new layer that will patch the previous one. More layers will be slightly slower in terms of I/O latency.

The COPY instruction copies the file from the build directory to a directory inside the container. The EXPOSE instruction sets the ports that will be accessible outside the container.

The VOLUME instruction indicates which directories can be mounted as host or data volumes on the container.

The CMD tells Docker which binary to execute when the container starts. In this case, we’ll tell it to execute the supervisor daemon, which then starts and monitors Gitlab execution.

The base image that we are using executes as foreground /assets/wrapper. That’s what we need to set on our supervisor.conf file:

[supervisord]
nodaemon = true

    [program: gitlab]
command = /assets/wrapper
numprocs = 1
autostart = true
autorestart = true

The nodaemon instructs the supervisord to execute in the foreground, and the second block tells the supervisor to execute GitLab’s boot script. autostart will tell it to start automatically when you run the container, and auto restart instructs the supervisor to restart GitLab automatically if it comes down for some reason.

So right now you should have the following files in your build directory:

• Dockerfile

• supervisor.conf

To build your image, execute this command in the build directory:

docker build - t yourcompany / gitlab - ce: 8.5.7 - ce.0.

Running the container

To launch containers, we recommend that you use docker-compose and write down a docker-compose.yml configuration file. Again it’s a better way to manage it as later on chances are that you won’t remember the settings you used to start your containing.

You can think of the Dockerfile as the place where you should describe the building blocks of your container and the docker-compose.yml as the place where you describe how a particular instance of the container should be executed and, if need be, where you override some configuration parameters.

gitlab:
    image: 'yourcompany/gitlab-ce:8.5.7-ce.0'
restart: always
hostname: 'gitlab.yourcompany.com'
environment:
    -GITLAB_OMNIBUS_CONFIG: |
    external_url 'https://gitlab.yourcompany.com'#
Add any other gitlab.rb configuration options
ports:
    -'443:443' - '22:22'
volumes:
    -'/docker/gitlab/conf:/etc/gitlab' - '/docker/gitlab/logs:/var/log/gitlab' - '/docker/gitlab/data:/var/opt/gitlab'

It’s very easy to understand what these options do.

You don’t need to pass Gitlab variables using this file, but it is a convenient way to override your default configuration if you are deploying multiple instances of this Docker image.

To run our container: docker-compose up -d

The first time it runs it will take some time to create all state like databases, etc.

If you are setting up Gitlab for the first time in Docker, this is it. Create your first user and start adding your mates. You can jump to the next section as well.

Importing existing Gitlab data

This step is easy as Gitlab already has a tool to export/import all data in bulk.

In your existing GitLab execute the steps below. It will only work if your previous setup is an Omnibus Package installation. If you installed it from the source, explore it to proceed. Make sure your Gitlab is running when you do this.

sudo gitlab - rake gitlab: backup: create

It can take some time depending on how many projects you have. In the end, it will write a file with a name like <unixtimestamp>_gitlab_backup.tar e.g. 1460911132.

To restore move this backup file into /docker/GitLab/data/backups. Then, enter the container by typing:

docker exec - it `docker ps | grep "yourcompany/gitlab-ce:8.5.7-ce.0" | cut -d " " -f1`
bash

Now enter these commands:

# stop services
gitlab-ctl stop unicorn
gitlab-ctl stop sidekiq
cd /var/opt/gitlab/backups
chown git:git *_gitlab_backup.tar
# Overwrite the contents of your GitLab database. It may take some time to complete, depending on how big your database is.
gitlab-rake gitlab:backup:restore BACKUP=<DATE> (e.g. 1460911132)
gitlab-ctl start
# Check if everything is ok. Watch out for errors and warnings
gitlab-rake gitlab:check SANITIZE=true

And that’s it! Access https://gitlab.yourcompany.com, log in with your user, and make sure everything is in there!

Backups

To automate backups it’s just a matter of running an instance of the docker-machine:

docker exec `docker ps | grep "yourcompany/gitlab-ce:8.5.7-ce.0" | cut -d " " -f1` / opt / gitlab / bin / gitlab - rake gitlab: backup: create

Install it in the host’s cron (say every day at 4 a.m.) using a more generic grep regex to avoid the need to change this cron job when you upgrade the version.

0 4 * * * docker exec `docker ps | grep "yourcompany/gitlab-ce:8.5.7-ce.0" | cut -d " " -f1` / opt / gitlab / bin / gitlab - rake gitlab: backup: create

This will only backup your database, so remember to backup your configuration files as well (/docker/GitLab/conf) and the logs in (/docker/GitLab/logs), in case you want to keep them.

Upgrading your Gitlab

First stop your current docker instance:

docker stop `docker ps | grep "yourcompany/gitlab-ce:8.5.7-ce.0" | cut -d " " -f1` 

Delete the container:

docker rm `docker ps -a | grep "yourcompany/gitlab-ce:8.5.7-ce.0" | cut -d " " -f1`

To upgrade you need to edit your Dockerfile and docker-compose.yml files. You only need to edit the tag, e.g. from 8.5.7-ce.0 to 8.5.8-ce.0. Build it again:

docker build - t yourcompany / gitlab - ce: 8.5.8 - ce.0.

And then start it:

docker - compose up - d

If all went well, you should now have a new version up and running. Please keep in mind that new versions might break stuff sometimes. So test it before deploying it and it can’t hurt to run gitlab-rake gitlab:check SANITIZE=true to check for issues.

Wrapping up

Migrating your Gitlab to Docker is the first step in turning your CI more manageable.

You can easily move it to another server if you need and you can easily upgrade to newer versions without running the risk of breaking things. Gitlab is the core of your team’s development infrastructure and you’ll want it to run smoothly. Docker can help you achieve just that.

Of course, if you want even more stability, you need a High-availability (HA) container setup (e.g. with HAProxy) or to set up live migration of containers and data.

Those are advanced topics that we may explore in a future blog post.

Further reading & Resources

• GitLab Docker images

• Backing up and restoring GitLab

• Data Sheet on JavaScript Security Threats