How to run mkpasswd with yescrypt on Ubuntu/Debian

Currently the Ubuntu/Debian mkpasswd command does not support yescrypt.

In order to use it anyway, we can use the ulikoehler/mkpasswd docker image to run the proper version of mkpasswd:

docker run --rm -it ulikoehler/mkpasswd

This will prompt you for a password and then echo the yescrypt encrypted and salted password:

$ docker run --rm -it ulikoehler/mkpasswd


Posted by Uli Köhler in Docker, Linux

How to use yum in Dockerfile correctly

Example of how to install the mkpasswd package using yum in your Dockerfile:

RUN yum -y install mkpasswd && yum -y clean all  && rm -rf /var/cache

There are two basic aspects to remember here:

  1. Use yum -y in order to avoid interactive Y/N questions during the automated build
  2. Use yum -y clean all && rm -rf /var/cache to clean up after the call to yum -y install

Complete Dockerfile example:

FROM fedora:34
RUN yum -y install mkpasswd && yum -y clean all  && rm -rf /var/cache


Posted by Uli Köhler in Container, Docker

How to fix docker.errors.DockerException: Error while fetching server API version: (‘Connection aborted.’, FileNotFoundError(2, ‘No such file or directory’))


While running a docker command like docker-compose pull, you see an error message like

Traceback (most recent call last):
  File "/usr/bin/docker-compose", line 33, in <module>
    sys.exit(load_entry_point('docker-compose==1.27.4', 'console_scripts', 'docker-compose')())
  File "/usr/lib/python3.8/site-packages/compose/cli/", line 67, in main
  File "/usr/lib/python3.8/site-packages/compose/cli/", line 123, in perform_command
    project = project_from_options('.', options)
  File "/usr/lib/python3.8/site-packages/compose/cli/", line 60, in project_from_options
    return get_project(
  File "/usr/lib/python3.8/site-packages/compose/cli/", line 131, in get_project
    client = get_client(
  File "/usr/lib/python3.8/site-packages/compose/cli/", line 41, in get_client
    client = docker_client(
  File "/usr/lib/python3.8/site-packages/compose/cli/", line 170, in docker_client
    client = APIClient(**kwargs)
  File "/usr/lib/python3.8/site-packages/docker/api/", line 197, in __init__
    self._version = self._retrieve_server_version()
  File "/usr/lib/python3.8/site-packages/docker/api/", line 221, in _retrieve_server_version
    raise DockerException(
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))


This means you haven’t started your docker service!

First, try to start it using

sudo systemctl start docker


sudo service docker start


sudo /etc/init.d/docker restart

(whatever works with your distribution).

After that, retry the command that originally caused the error message to appear.

In case it still shows the same error message, try the following steps:

  • First, check /var/log/docker.log using
    cat /var/log/docker.log

    Check that file for errors during docker startup.

  • Also check if the user you’re running the command as is a member of the docker group. While insufficient permissions will not cause a FileNotFoundError(2, 'No such file or directory')), but a Permission denied, the error message might look similar in some cases.
Posted by Uli Köhler in Container, Docker, Linux

How to fix Synology Docker: failed to initialize logging driver: database is locked


When you try to start a specific Docker container using the Synology NAS GUI, the container is being stopped unexpectedly and you see an error message like this in the logs:

Start container mycontainer failed: {"message":"failed to initialize logging driver: database is locked"}.
Signal container mycontainer failed: {"message":"Cannot kill container: mycontainer: Container 5136ddceeb46004c5b18f04eb9ec10cac3808938515874fc31185b0964232201 is not running"}.


I fixed this problem by stopping the container, duplicating the container session: Right click on the container -> Settings -> Duplicate Settings

That will create a new container with the given settings. Note that local ports will be set to Auto and will not be copied over, so if you use fixed local ports, you need to set them to a different value in the original container and then set the local ports on the new container to the desired fixed value. Also note that files inside the container are not copied over. In my configuration, all relevant files are stored in mapped volumes on the NAS.

The root cause of this issue seems to be that the logging database for this specific container has been locked by some process. The issue is always limited to a certain container and will not affect other containers (though it could in principle occur for more than one container). I know that at least in my specific case, the issue is not caused by a reboot and will also not be fixed by a reboot of the Synology NAS. Just before I encountered the issue, my NAS had not been rebooted for months, but it might be related to Synology package updates since I updated some packages using the Package manager just before encountering the issue, including a Synology Mail Plus update which failed on the first attempt, but succeeded when I clicked Update again.

Posted by Uli Köhler in Docker, Networking

A modern Kimai setup using docker-compose and nginx

This is the setup I use to run multiple productive kimai instances. In my example, I create the files in /opt/kimai-mydomain. The folder name is not critical, but it is helpful to distinguish multiple indepedent kimai instances.

First, let’s create /opt/kimai-mydomain/docker-compose.yml. You don’t need to modify anything in this file as every relevant configuration is loaded from .env using environment variables.

version: '3.5'
    image: mariadb:latest
      - MYSQL_DATABASE=kimai
      - MYSQL_USER=kimai
      - ./mariadb_data:/var/lib/mysql
    command: --default-storage-engine innodb
    restart: unless-stopped
      test: mysqladmin -p${MARIADB_ROOT_PASSWORD} ping -h localhost
      interval: 20s
      start_period: 10s
      timeout: 10s
      retries: 3

    image: kimai/kimai2:apache-debian-master-prod
      - APP_ENV=prod
      - TRUSTED_HOSTS=localhost,${HOSTNAME}
      - [email protected]
      - DATABASE_URL=mysql://kimai:${MARIADB_PASSWORD}@mariadb/kimai
      - ./kimai_var:/opt/kimai/var
      - '17919:8001'
      - mariadb
    restart: unless-stopped

Now we’ll create the configuration in /opt/kimai-mydomain/.env:

[email protected]

Generate random passwords for .env ! Do NOT leave the default passwords in .env !

You also need to set KIMAI_ADMIN_EMAIL and HOSTNAME correctly.

We can now create the kimai data directory and set the correct permissions:

mkdir -p kimai_var
chown -R 33:33 kimai_var

(33 is the user ID and group ID of the www-data user inside the container)

Now, we will initialize the kimai database and the user:

docker-compose run kimai console kimai:install -n

Once you see a line like

[Sun Mar 07 23:53:35.986477 2021] [core:notice] [pid 50] AH00094: Command line: '/usr/sbin/apache2 -D FOREGROUND'

stop the process using Ctrl+C as this means that Kimai has finished installing.

Now we can create a systemd service that automatically starts Kimai using TechOverflow’s method from Create a systemd service for your docker-compose project in 10 seconds:

curl -fsSL | sudo bash /dev/stdin

Now we only need to create an nginx config for reverse proxying of your Kimai domain. There is nothing special to be considered for the config, hence I’ll show my config just as an example that you can copy and paste.

server {

    location / {
        proxy_pass http://localhost:17919/;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_redirect default;

    listen [::]:443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot

server {
    if ($host = {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    listen [::]:80; # managed by Certbot
    return 404; # managed by Certbot

After setting up your config – I always recommend to setup TLS using Let’s Encrypt, even for test setups, open your Browser and go to your Kimai domain, e.g. to You can directly login to kimai using KIMAI_ADMIN_EMAIL and KIMAI_ADMIN_PASSWORD as specified in .env.

Posted by Uli Köhler in Container, Docker

How to install python3 pip / pip3 in Alpine Linux


You want to install pip3 (also called python3-pip) in Alpine linux, but running apk install python3-pip shows you that the package doesn’t exist

/ # apk add python3-pip
ERROR: unable to select packages:
  python3-pip (no such package):
    required by: world[python3-pip]


You need to install py3-pip instead using

apk add py3-pip

Example output:

/ # apk add py3-pip
(1/35) Installing libbz2 (1.0.8-r1)
(2/35) Installing expat (2.2.10-r1)
(3/35) Installing libffi (3.3-r2)


Posted by Uli Köhler in Alpine Linux, Container, Docker, Linux

A modern Docker-Compose config for Etherpad using nginx as reverse proxy

This is the configuration I use to run my etherpad installations behind nginx:


version: "3.5"
    image: etherpad/etherpad:latest
      - TITLE=My Etherpad
      - DEFAULT_PAD_TEXT=My Etherpad
      - ADMIN_USER=admin
      - DB_TYPE=mysql
      - DB_HOST=mariadb
      - DB_PORT=3306
      - DB_USER=etherpad
      - DB_NAME=etherpad
      - DB_CHARSET=utf8mb4
      - SESSION_REQUIRED=false
      - "17201:9001"
      - mariadb

    image: mariadb:latest
      - MYSQL_DATABASE=etherpad
      - MYSQL_USER=etherpad
      - './mariadb_data:/var/lib/mysql'
    command: ['mysqld', '--character-set-server=utf8mb4', '--collation-server=utf8mb4_unicode_ci']
      test: mysqladmin -p${MARIADB_ROOT_PASSWORD} ping -h localhost
      interval: 20s
      start_period: 10s
      timeout: 10s
      retries: 3


Remember to replace the passwords with your own random passwords !


nginx config

server {
    access_log off;
    error_log /var/log/nginx/;

    location / {
        proxy_pass http://localhost:17201/;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_redirect default;

    listen [::]:443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot


server {
    if ($host = {
        return 301 https://$host$request_uri;
    } # managed by Certbot

    listen [::]:80; # managed by Certbot
    return 404; # managed by Certbot

How to autostart

Our post Create a systemd service for your docker-compose project in 10 seconds provides an extremely quick and easy-to-use one-liner to create and autostart a systemd service running docker-compose for your etherpad instance.

How to backup

This will be detailed in a future blogpost.

Posted by Uli Köhler in Container, Docker, nginx

How to use apt install correctly in your Dockerfile

This is the correct way to use apt install in your Dockerfile:

ENV DEBIAN_FRONTEND=noninteractive
RUN apt update && apt install -y PACKAGE && rm -rf /var/lib/apt/lists/*

Key takeaways:

  • Set DEBIAN_FRONTEND=noninteractive to prevent some packages from prompting interactive input (tzdata for example), which leads to indefinite waiting for an user input
  • Run apt update before the install command to fetch the current package lists
  • apt install with -y to prevent apt from asking you if you really want to install the packages
  • rm -rf /var/lib/apt/lists/* after the install command in order ot prevent the cached apt lists (which are fetched by apt update) from ending up in the container image
  • All of that in one command joined by && in order to prevent docker from building separate layers for each part of the command (and to prevent it from first storing /var/lib/apt/lists in one layer and then delete it in another layer)

Also see the official guide on Dockerfile best practices

Posted by Uli Köhler in Container, Docker

How to fix Gitlab CI error during connect: Post http://docker:2375/v1.40/auth: dial tcp: lookup docker on … no such host


In your GitLab CI jobs, you see this error message while trying to run some docker command:

error during connect: Post http://docker:2375/v1.40/auth: dial tcp: lookup docker on no such host


This error occurs with docker-based gitlab runners such as the one we’re that are configured using a docker executor. The error message means that the inner docker container doesn’t have a connection to the host docker daemon.

In order to fix this, set these options in [runners.docker] in your gitlab-runner config.toml:

privileged = true
volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"]

A valid config.toml looks like this:

concurrent = 1
check_interval = 0

  session_timeout = 1800

  name = "CoreOS-Haar-Runner"
  url = ""
  token = "bemaiBie8usahMoo9ish"
  executor = "docker"
    tls_verify = false
    image = "docker:stable"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"]
    shm_size = 0

Why this works

Adding "/var/run/docker.sock:/var/run/docker.sock" allows the inner docker container running inside the gitlab-runner to access the outer (host) docker daemon. Without this, it tries to connect to the docker default URL, i.e. http://docker:2375. This URL can’t be resolved via DNS (the DNS server in my case is, the DNS standard port being 53), hence docker prints the error message listed above.

Posted by Uli Köhler in Docker

How to enable SSH password authentication on Fedora CoreOS

Add this config line to your Fedora CoreOS Ignition config in order to enable SSH password authentication on your install.

    - path: /etc/ssh/sshd_config.d/20-enable-passwords.conf
      mode: 0644
        inline: |
          PasswordAuthentication yes

By default, Fedora CoreOS will only allow pubkey authentication and disable password authentication. This Ignition config will set PasswordAuthentication yes as a config option for the SSH daemon.

Original source: The Fedore CoreOS authentication guide.

Posted by Uli Köhler in CoreOS

How to install fcct on Ubuntu Linux

Note: This currently doesn’t work due to libc issues on most Ubuntu versions. I am working on a solution !

I use fcct, the Fedora CoreOS Configuration Transpiler in order to create Ignition JSON files for installing CoreOS from YAML

Use these commands in your shell to download fcct to /usr/local/bin/fcct and hence install it. This works on most Linux distributions, but I mainly use it in Ubuntu and Debian.

sudo wget -O /usr/local/bin/fcct
chmod a+x /usr/local/bin/fcct

See Fedora CoreOS minimal ignition config for XCP-NG for my Fedora CoreOS ignition config

Posted by Uli Köhler in Container

How to install Kompose on Ubuntu in 10 seconds

Kompose is a tool that converts docker-compose.yml into Kubernetes YAML config files.

You can easily install it globally on most Linux distributions using this command:

sudo curl -L -o /usr/local/bin/kompose
chmod a+x /usr/local/bin/kompose

After that, run

kompose convert

in a directory containing a docker-compose.yml. See the Kompose website for further information.

Posted by Uli Köhler in Container, Docker, Kubernetes, Podman

How to install Podman on Ubuntu 20.04 in 25 seconds

Run this one-liner to install podman on your Ubuntu system:

wget -qO- | sudo bash /dev/stdin

This is the code that is being run, which is exactly the code taken from the Podman installation page.

. /etc/os-release
echo "deb${VERSION_ID}/ /" | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list
curl -L${VERSION_ID}/Release.key | sudo apt-key add -
sudo apt-get update
sudo apt-get -y upgrade
sudo apt-get -y install podman


Posted by Uli Köhler in Container, Linux, Podman

Minimal Dockerfile example

This is a minimal Dockerfile that you can use to build a test container:

FROM alpine
CMD ["/bin/sh", "-c", "echo 'It works!'"]

Build example:

docker build -t techoverflow-minimal-docker .

Run example:

docker run techoverflow-minimal-docker

Example output:

$ docker run techoverflow-minimal-docker       
It works!


Posted by Uli Köhler in Docker

Best-practice configuration for MongoDB with docker-compose

Create /var/lib/mongodb/docker-compose.yml:

version: '3.1'
    image: mongo
        - ./data:/data/db
        - 27017:27017
    command: --serviceExecutor adaptive

This will store the MongoDB data in /var/lib/mongodb/data. I prefer this variant to using docker volumes since this method keeps all MongoDB-related data in the same directory.

Then create a systemd service using

curl -fsSL | sudo bash /dev/stdin

See our post on how to Create a systemd service for your docker-compose project in 10 seconds for more details on this method.

You can access MongoDB at localhost:27017! It will autostart after boot

Restart by

sudo systemctl restart mongodb

Stop by

sudo systemctl stop mongodb

View logs:

sudo journalctl -xfu mongodb

View logs in less:

sudo journalctl -xu mongodb


Posted by Uli Köhler in Docker, MongoDB

How to disable SELinux in Fedora CoreOS

Warning: Depending on your application, disabling the SELinux security layer might be a bad idea since it might introduce new security risks especially if the container system has security issues.

In order to disable SELinux on Fedora CoreOS, run this:

sudo sed -i -e 's/SELINUX=/SELINUX=disabled #/g' /etc/selinux/config
sudo systemctl reboot

Note that this will reboot your system in order for the changes to take effect.

The sed command will replace the default


in /etc/selinux/config to



Posted by Uli Köhler in Container, Docker, Linux

How to export gitlab backup in Gitlab Docker container

GitLab provides an integrated feature to export a TAR file containing all data from the current GitLab instance.

How to make a GitLab backup?

In order to run this for a GitLab instance running on docker-compose use

docker-compose exec gitlab gitlab-backup create STRATEGY=copy

where gitlab is the container running the gitlab image, e.g. gitlab/gitlab-ce:latest.

In case you are running a docker-based setup without docker-compose, run

docker exec my-gitlab-container gitlab-backup create STRATEGY=copy

where my-gitlab-container is the ID or name of the container

Where to find the backups?

By default, gitlab stores the backups in /var/opt/gitlab/backups. In case you need to change this, look for the following line in /etc/gitlab/gitlab.rb:

gitlab_rails['backup_path'] = "/var/opt/gitlab/backups"

In my docker-compose configuration, I map out the entire /var/opt/gitlab directory:

   - './data:/var/opt/gitlab'

hence I can find the backups in ./data/backups:

$ ls data/backups/
Posted by Uli Köhler in Docker

How to create a systemd backup timer & service in 10 seconds

In our previous post Create a systemd service for your docker-compose project in 10 seconds we introduced a script that automatically creates a systemd service to start a docker-compose-based project. In this post, we’ll show

First, you need to create a file named in the directory where docker-compose.yml is located. This file will be run by the systemd service every day. What that file contains is entirely up to you and we will provide examples in future blogposts.

Secondly, run

wget -qO- | sudo bash /dev/stdin

from the directory where docker-compose.yml is located. Note that the script will use the directory name as a name for the service and timer that is created. For example, running the script in /var/lib/redmine-mydomain will cause redmine-mydomain-backup to be used a service name.

Example output from the script:

Creating systemd service... /etc/systemd/system/redmine-mydomain-backup.service
Creating systemd timer... /etc/systemd/system/redmine-mydomain-backup.timer
Enabling & starting redmine-mydomain-backup.timer
Created symlink /etc/systemd/system/ → /etc/systemd/system/redmine-mydomain-backup.timer.

The script will create /etc/systemd/systemd/redmine-mydomain-backup.service containing the specification on what exactly to run:



and /etc/systemd/systemd/redmine-mydomain-backup.timer containing the logic when the .service is started:




and will automatically start and enable the timer. This means: no further steps are needed after running this script!

In order to show the current status of the service, use e.g.

sudo systemctl status redmine-mydomain-backup.timer

Example output:

● redmine-mydomain-backup.timer - redmine-mydomain-backup
     Loaded: loaded (/etc/systemd/system/redmine-mydomain-backup.timer; enabled; vendor preset: enabled)
     Active: active (waiting) since Thu 2020-12-10 02:50:31 CET; 19min ago
    Trigger: Fri 2020-12-11 00:00:00 CET; 20h left
   Triggers: ● redmine-mydomain-backup.service

Dec 10 02:50:31 myserverhostname systemd[1]: Started redmine-mydomain-backup.

In the

Trigger: Fri 2020-12-11 00:00:00 CET; 20h left

line you can see when the service will be run next. By default, the script generates tasks that run OnCalendar=daily, which means the service will be run on 00:00:00 every day. Checkout the systemd.time manpage for further information on the syntax you can use to specify other timeframes.

In order to run the backup immediately (it will still run daily after doing this), do

sudo systemctl start redmine-mydomain-backup.service

(note that you need to run systemctl start on the .service! Running systemctl start on the .timer will only enable the timer and not run the service immediately).

In order to view the logs, use

sudo journalctl -xfu redmine-mydomain-backup.service

(just like above, you need to run journalctl -xfu on the .service, not on the .timer).

In order to disable automatic backups, use e.g.

sudo systemctl disable redmine-mydomain-backup.timer

Source code:

# Create a systemd service & timer that runs the given backup daily
# by Uli Köhler -
# Licensed as CC0 1.0 Universal
export SERVICENAME=$(basename $(pwd))-backup

export SERVICEFILE=/etc/systemd/system/${SERVICENAME}.service
export TIMERFILE=/etc/systemd/system/${SERVICENAME}.timer

echo "Creating systemd service... $SERVICEFILE"
sudo cat >$SERVICEFILE <<EOF


echo "Creating systemd timer... $TIMERFILE"
sudo cat >$TIMERFILE <<EOF



echo "Enabling & starting $SERVICENAME.timer"
sudo systemctl enable $SERVICENAME.timer
sudo systemctl start $SERVICENAME.timer


Posted by Uli Köhler in Docker, Linux

How to use pg_dump in Gitlab Docker container

When using the offical gitlab Docker container, you can use this command to run psql:

docker exec -t -u gitlab-psql [container name] pg_dump -h /var/opt/gitlab/postgresql/ -d gitlabhq_production > gitlab-dump.sql

This will save the SQL dump of the database into gitlab-dump.sql

In case you’re using a docker-compose based setup, use this command:

docker-compose exec -u gitlab-psql gitlab pg_dump -h /var/opt/gitlab/postgresql/ -d gitlabhq_production > gitlab-dump.sql

Note that gitlab in this command is the container name.

Posted by Uli Köhler in Docker, Linux