Docker

How to easily install Redmine using Docker Images

Note: Also see this followup post on how to use custom themes in this setup and this followup post on how to backup Redmine using this setup.

This tutorial shows you step-by-step the easiest method of setting up a fresh redmine installation I have found so far. The commands have been tested on Ubuntu 18.04, but they should work with minimal modification on other DEB-based distributions

Installing Docker & Docker-Compose

Please follow the instructions in How to install docker and docker-compose on Ubuntu in 30 seconds

Creating the directory & docker-compose configuration

We will install redmine in /var/lib/redmine which will host the data directories and the docker-compose script.

# Create directories
sudo mkdir /var/lib/redmine
sudo mkdir -p /var/lib/redmine/redmine_data /var/lib/redmine/mariadb_data
# Set correct permissions for the directories
sudo chown -R $USER:docker /var/lib/redmine
sudo chown -R 1001:1001 /var/lib/redmine/redmine_data /var/lib/redmine/mariadb_data

Next, we’ll create /var/lib/redmine/docker-compose.yml.

There’s a couple of things you need to change here:

  • Set REDMINE_EMAIL to the email of the admin user you want to use (usually that is your email!)
  • Set the SMTP credentials (SMTP_HOSTSMTP_PORTSMTP_USER and SMTP_PASSWORD) to a valid SMTP server. SMTP_TLS defaults to true – in the rare case that
  • The ports configuration, in this case '3718:3000' means that Redmine will be mapped to port 3718 on the local PC. This port is chosen somewhat arbitarily – as we will run redmine behind an nginx reverse proxy, the port does not need to be any port in particular (as long as you use the same port everywhere), but it may not be used by anything else. You can use any port here, provided that it’s not used for anything else. Leave 3000 as-is and only change 3718 if required.

Note that you do not need to change REDMINE_PASSWORD – when you login for the first time, redmine will force you to change the password anyway.

version: '2'
services:
  mariadb:
    image: 'bitnami/mariadb:latest'
    environment:
      - ALLOW_EMPTY_PASSWORD=yes
    volumes:
      - '/var/lib/redmine/mariadb_data:/bitnami'
  redmine:
    image: 'bitnami/redmine:latest'
    environment:
      - REDMINE_USERNAME=admin
      - REDMINE_PASSWORD=redmineadmin
      - [email protected]
      - SMTP_HOST=smtp.gmail.com
      - SMTP_PORT=25
      - [email protected]
      - SMTP_PASSWORD=yourGmailPassword
    ports:
      - '3718:3000'
    volumes:
      - '/var/lib/redmine/redmine_data:/bitnami'
    depends_on:
      - mariadb

Setting up the systemd service

Next, we’ll configure the systemd service in /etc/systemd/system/redmine.service.

Set User=... to your current user in the [Service] section.

[Unit]
Description=Redmine
Requires=docker.service
After=docker.service

[Service]
Restart=always
User=uli
Group=docker
# Shutdown container (if running) when unit is stopped
ExecStartPre=/usr/local/bin/docker-compose -f /var/lib/redmine/docker-compose.yml down -v
# Start container when unit is started
ExecStart=/usr/local/bin/docker-compose -f /var/lib/redmine/docker-compose.yml up
# Stop container when unit is stopped
ExecStop=/usr/local/bin/docker-compose -f /var/lib/redmine/docker-compose.yml down -v

[Install]
WantedBy=multi-user.target

After creating the file, we can enable and start the redmine service:

sudo systemctl enable redmine
sudo systemctl start redmine

The output of sudo systemctl start redmine should be empty. In case it is

Job for redmine.service failed because the control process exited with error code.
See "systemctl status redmine.service" and "journalctl -xe" for details.

debug the issue using journalctl -xe and journalctl -e

The first startup usually takes about 3 minutes, so grab a cup of coffee.

Now you can check if redmine is running using

wget -qO- http://localhost:3718/

(if you changed the port config before, you need to use your custom port in the URL).

If it worked, it will show a large HTML output, ending with

[...]
<div id="footer">
  <div class="bgl"><div class="bgr">
    Powered by <a href="https://www.redmine.org/">Redmine</a> &copy; 2006-2018 Jean-Philippe Lang
  </div></div>
</div>
</div>
</div>

</body>
</html>

If the output is empty, try wget -O- http://localhost:3718/ to see the error message

Setting up the nginx reverse proxy (optional but recommended)

We’ll use nginx to proxy the requests from a certain domain (Using Apache, if you use it already, is also possible but it is outside the scope of this tutorial to tell you how to do that). Install it using

sudo apt -y install nginx

First, you’ll need a domain name with DNS being configured. For this example, we’ll assume that your domain name is redmine.techoverflow.net ! You need to change it to your domain name!

First, we’ll create the config file in /etc/nginx/sites-enabled/redmine.conf. Remember to replace redmine.techoverflow.net by your domain name! If you use a port different from 3718, replace that as ewll.

server {
    listen 80;
    server_name redmine.techoverflow.net;

    access_log /var/log/nginx/redmine.access_log;
    error_log /var/log/nginx/redmine.error_log info;

    location / {
        proxy_pass http://127.0.0.1:3718; # docker-compose forwarded
        proxy_read_timeout 3600s;
        proxy_http_version 1.1;
    }

}

Now run sudo nginx -t to test if there are any errors in the config file. If everything is alright, you’ll see

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Once you have fixed all errors, if any, run sudo service nginx reload to apply the configuration.

Test the setup by navigating your domain name in the browser. You should see the redmine interface:

Securing the nginx reverse proxy using Let’s Encrypt

First we need to install certbot and the certbot nginx plugin in order to create & install the certificate in nginx:

sudo apt -y install python3-certbot python3-certbot-nginx

Fortunately certbot automates most of the process of installing & configuring SSL and the certificate. Run

sudo certbot --nginx

It will ask you to enter your Email address and agree to the terms of service and if you want to receive the EFF newsletter.

After that, certbot will ask you to select the correct domain name:

Which names would you like to activate HTTPS for?
-------------------------------------------------------------------------------
1: redmine.techoverflow.net
-------------------------------------------------------------------------------
Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel):

In this case, there is only one domain name (there will be more if you have more domains active on nginx!).

Therefore, enter 1 and press enter. certbot will now generate the certificate. In case of success you will see

Deploying Certificate to VirtualHost /etc/nginx/sites-enabled/redmine.techoverflow.net.conf

Now it will ask you whether to redirect all requests to HTTPS automatically:

Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access.
-------------------------------------------------------------------------------
1: No redirect - Make no further changes to the webserver configuration.
2: Redirect - Make all requests redirect to secure HTTPS access. Choose this for
new sites, or if you're confident your site works on HTTPS. You can undo this
change by editing your web server's configuration.
-------------------------------------------------------------------------------
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 

Choose Redirect here: Type 2 and press enter. Now you can login to redmine and finish the installation.

You need to renew the certificate every 3 months for it to stay valid, and run sudo service nginx reload afterwards to use the new certificate. If you fail to do this, users will see certificate expired error messages and won’t be able to access Redmine easily! See this post for details on how to mostly automate this process!

Setting up Redmine

Go to your domain name (if you have followed the instructions above, it should automatically redirect you to HTTPS). Click Login at the top right and login with the username admin and the default password redmineadmin. Upon first login, it will require you to change the password to a new – and more secure password.

I won’t describe in detail how to setup Redmine for your project. However there’s two things you should take care of immediately after the first login:

  1. Configure the correct domain name: Go to Administration -> Settings and set Host name and path to your domain name, e.g. redmine.techoverflow.net. Set Protocol to HTTPS. You can also set a custom name for your Redmine installation under Application Title
  2. Still under Administration -> Settings, go to the Email Notifications tab, set an approriate sender email address under Emission email address (usually you would use [email protected] here, but you might want to use your SMTP username for some SMTP providers like GMail)
  3. Scroll down to the bottom of the Email Notifications page and click Send a test email which will send a test email to the current redmine user’s email adress. Unless you have changed it, the default is the address configured in REDMINE_EMAIL in /var/lib/redmine/docker-compose.yml.

In case the email does not work, change SMTP_...=... in /var/lib/redmine/docker-compose.yml but you also have to change it in /var/lib/redmine/redmine_data/redmine/conf/configuration.yml ! After doing the changes, restart redmine by

sudo systemctl restart redmine

which will use the new configuration from the config file.

Block access to the forwarded port using ufw (optional)

ufw is a simple Firewall for Ubuntu. Use sudo apt install ufw to install it and sudo ufw enable to activate it. The default configuration will allow SSH but it will block other ports, including port 3718 or any other custom port you might have used.

In order to enable it, use

sudo ufw enable
sudo ufw allow ssh
sudo ufw allow http
sudo ufw allow https

Remember to add any ports you need to have open to the list as well. See the ufw docs for more information.

Posted by Uli Köhler in Container, Docker, Linux, Project management

How to run ‘docker-compose up’ in the background

In order to run docker-compose up in the background, use

docker-compose up -d

The -d option means --detach, i.e. the process is detached from the foreground shell you are running.

Posted by Uli Köhler in Container, Docker, Linux

Solving Bitnami Docker Redmine ‘cannot create directory ‘/bitnami/mariadb’: Permission denied’

Problem:

You are setting up a docker-based redmine installation using the bitnami image, but you’re getting this error message when you use a host directory mounted as volume:

cannot create directory '/bitnami/mariadb': Permission denied

Solution:

Run

sudo chown -R 1001:1001 <directory>

on the host directories used by both the MariaDB container and the Redmine container.

In order to find the directories, look for these lines in the docker-compose YML file::

# Example: This can be found in the mariadb section:
    volumes:
      - '/var/lib/myredmine/mariadb_data:/bitnami'
# Example: This can be found in the redmine section
    volumes:
      - '/var/lib/myredmine/redmine_data:/bitnami'

In this example, you would have to run

sudo chown -R 1001:1001 /var/lib/myredmine/mariadb_data /var/lib/myredmine/redmine_data

and then restart the container:

docker-compose down
docker-compose up # Use 'docker-compose up -d' to run in the background

 

Posted by Uli Köhler in Container, Docker, Linux

A systemd service template for docker-compose

Here’s my template for running a docker-compose service as a systemd service:

# Save as e.g. /etc/systemd/system/my-service.service
[Unit]
Description=MyService
Requires=docker.service
After=docker.service

[Service]
Restart=always
User=uli
Group=docker
# Shutdown container (if running) when unit is stopped
ExecStartPre=/usr/local/bin/docker-compose -f /home/uli/mydockerservice/docker-compose.yml down -v
# Start container when unit is started
ExecStart=/usr/local/bin/docker-compose -f /home/uli/mydockerservice/docker-compose.yml up
# Stop container when unit is stopped
ExecStop=/usr/local/bin/docker-compose -f /home/uli/mydockerservice/docker-compose.yml down -v

[Install]
WantedBy=multi-user.target

In order to get it up and running for your application, you need to modify a couple of things:

  1. Check if you have docker-compose in /usr/local/bin/docker-compose (as I do, because I use the docker-ce installation from the official docker repositories for Ubuntu 18.04) or in /usr/bin/docker-compose (in which case you need to set the correct docker-compose path in all 3 places in the service file)
  2. Ensure that the user you want to run docker-compose as (uli in this example) is a member of the docker group (sudo usermod -a -G docker <user>), and set the correct user in the User=... line
  3. Define a name for your service that should be reflected in both the service filename and the Description=... line
  4. Set the correct path for your docker-compose YML config file in all the Exec…=… lines (i.e. replace /home/uli/mydockerservice/docker-compose.yml by your YML path).

After that, you can start your service using

sudo systemctl start my-service # --> my-service.service, use whatever you named your file as

and optionally enable it at bootup:

systemctl enable docker # Docker is required for your service so you need to enable it as well!
systemctl enable my-service # --> my-service.service, use whatever you named your file as
Posted by Uli Köhler in Container, Docker, Linux

How to fix docker ‘Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?’ on Ubuntu

Problem:

You’re running a docker command like docker ps, but you only see this error message:

Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

Solution:

As the error message already tells you, the docker daemon is currently not running.

On Ubuntu (16.04 upwards) and many other systemd-based distributions, you can fix this by

sudo systemctl start docker

In most cases, you want to automatically start the docker daemon at boot. In order to do this, run

sudo systemctl enable docker

After that, run your command (e.g. docker ps) again.

Posted by Uli Köhler in Container, Docker, Linux

How to fix docker ‘Got permission denied while trying to connect to the Docker daemon socket’

Problem:

You are running a command like docker ps but you get this error message:

Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.39/containers/json: dial unix /var/run/docker.sock: connect: permission denied

Solution:

As a quick fix, running the command as root using sudo (e.g. sudo docker ps) will solve the issue temporarily.

The issue here is that the user you’re running the command as is not a member of the docker group. In order to add it to the docker group, run

sudo usermod -a -G docker $USER

After running that command, you need to logout and log back in to your computer (or terminate your SSH session and re-connect in case you are logged in using SSH) – else, the group change does not take effect.

Running groups should show you that you now belong to the docker group:

$ groups
uli sudo www-data lxd docker # Check if docker appears here!

After that, retry running the command (e.g. docker ps) – the error should now have disappeared.

See What does sudo usermod -a -G group $USER do on Linux? for details on what this command changes on your system and what the parameters mean.

Background information

When you run any docker command on Linux, the docker binary will try to connect to /var/run/docker.sock. This allows you to run docker commands as non-root-user without using sudo all the time.

When you start the docker daemon, it will create /var/run/docker.sock as a unix socket for client applications to connect to.

You can have a look at the owner, group and permissions of the docker socket by using stat /var/run/docker.sock on the command line:

$ stat /var/run/docker.sock
  File: /var/run/docker.sock
  Size: 0               Blocks: 0          IO Block: 4096   socket
Device: 16h/22d Inode: 677         Links: 1
Access: (0660/srw-rw----)  Uid: (    0/    root)   Gid: (  999/  docker)
Access: 2019-04-30 01:32:21.718150679 +0200
Modify: 2019-04-24 18:37:39.236357175 +0200
Change: 2019-04-24 18:37:39.240357175 +0200
 Birth: -

For our purposes, the interesting information is Uid: ( 0/ root) Gid: ( 999/ docker) which tells you that the docker socket is owned by the user root and the group docker. The group ID might be different on your computer, but only the name of the group is relevant.

Given the permissions Access: (0660/srw-rw----), both the owner (root) and the group (docker) can read & write (rw) to the docker socket. This means that if you are either the user root (which you can become temporarily using sudo) or you are a member of the docker group, you will be able to connect to that socket and communicate with the docker daemon.

Note that the docker daemon itself (dockerd) is running as root, which you can check using

$ ps aux | grep dockerd
root      2680  0.1  0.3 1247872 19828 ?       Ssl  Apr24   7:44 /usr/bin/dockerd -H fd://

For more information on the docker daemon, see the official Docker daemon guide.

Posted by Uli Köhler in Container, Docker, Linux

How to list all currently running docker containers?

To list all currently running docker containers run

docker ps

If no containers are running, this will give you this output:

CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

In case some containers are running, there will be additional lines listing the container like

CONTAINER ID        IMAGE                    COMMAND                  CREATED             STATUS              PORTS                    NAMES
1bd0a1461b38        bitnami/mariadb:latest   "/entrypoint.sh /run…"   6 minutes ago       Up 6 minutes        3306/tcp                 mydb

 

Posted by Uli Köhler in Container, Docker, Linux

Solving Docker permission denied while trying to connect to the Docker daemon socket

Problem:

You are trying to run a docker container or do the docker tutorial, but you only get an error message like this:

docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.26/containers/create: dial unix /var/run/docker.sock: connect: permission denied.
See 'docker run --help'.

Solution:

The error message tells you that your current user can’t access the docker engine, because you’re lacking permissions to access the unix socket to communicate with the engine.

As a temporary solution, you can use sudo to run the failed command as root (e.g. sudo docker ps).
However it is recommended to fix the issue by adding the current user to the docker group:

Run this command in your favourite shell and then completely log out of your account and log back in (or exit your SSH session and reconnect, if in doubt, reboot the computer you are trying to run docker on!):

sudo usermod -a -G docker $USER

After doing that, you should be able to run the command without any issues. Run docker run hello-world as a normal user in order to check if it works. Reboot if the issue still persists.

See What does sudo usermod -a -G group $USER do on Linux? for details on what this command changes on your system and what the parameters mean.

Logging out and logging back in is required because the group change will not have an effect unless your session is closed.

Background information

On Linux, when you run any docker command, the docker binary will try to connect to /var/run/docker.sock. As indicated by its .sock extension, this file is a Unix Domain Socket – basically, a way so multiple processes can communicate on the local computer (also called an IPC mechanism – IPC = “Inter-Process Communication”).

In the case of Docker, the main reason for using the socket is that any user belonging to the docker group can connect to the socket while the Docker daemon itself can run as root. Essentially, it’s a convenience feature and allows multiple docker client commands to communicate to the same daemon process internally.

Posted by Uli Köhler in Container, Docker, Linux

Docker: Remove all images and containers

Problem:

You use Docker, but working with it created lots of images and containers. You want to remove all of them to save disk space.

Continue reading →

Posted by Uli Köhler in Container, Docker