Docker

How to install ruby & rubygems in Alpine Linux

Problem:

You want to install ruby and the gem package manager in Alpine linux, but running apk install ruby rubygems shows you that the package doesn’t exist

/ # apk add ruby rubygems
ERROR: unable to select packages:
  rubygems (no such package):
    required by: world[rubygems]

Solution:

gem is included in the ruby package. So the only command you need to run is

apk update
apk add ruby

Example output:

/ # apk add ruby
(1/7) Installing ca-certificates (20191127-r5)
(2/7) Installing gdbm (1.19-r0)
(3/7) Installing gmp (6.2.1-r0)
(4/7) Installing readline (8.1.0-r0)
(5/7) Installing yaml (0.2.5-r0)
(6/7) Installing ruby-libs (2.7.3-r0)
(7/7) Installing ruby (2.7.3-r0)
Executing busybox-1.32.1-r6.trigger
Executing ca-certificates-20191127-r5.trigger
OK: 928 MiB in 154 packages

After doing that, you can immediately use both ruby and gem.

Posted by Uli Köhler in Alpine Linux, Container, Docker, Linux, Ruby

How to run mkpasswd with yescrypt on Ubuntu/Debian

Currently the Ubuntu/Debian mkpasswd command does not support yescrypt.

In order to use it anyway, we can use the ulikoehler/mkpasswd docker image to run the proper version of mkpasswd:

docker run --rm -it ulikoehler/mkpasswd

This will prompt you for a password and then echo the yescrypt encrypted and salted password:

$ docker run --rm -it ulikoehler/mkpasswd
Password:
$y$j9T$YzrfO5lQkDWahpz5pwYzg/$HzQoMYt.7E1jj.sd6OyYCGI/Qk6oGehNgz5uvY1qp59

 

Posted by Uli Köhler in Docker, Linux

How to use yum in Dockerfile correctly

Example of how to install the mkpasswd package using yum in your Dockerfile:

RUN yum -y install mkpasswd && yum -y clean all  && rm -rf /var/cache

There are two basic aspects to remember here:

  1. Use yum -y in order to avoid interactive Y/N questions during the automated build
  2. Use yum -y clean all && rm -rf /var/cache to clean up after the call to yum -y install

Complete Dockerfile example:

FROM fedora:34
RUN yum -y install mkpasswd && yum -y clean all  && rm -rf /var/cache

 

Posted by Uli Köhler in Container, Docker

How to fix docker.errors.DockerException: Error while fetching server API version: (‘Connection aborted.’, FileNotFoundError(2, ‘No such file or directory’))

Problem:

While running a docker command like docker-compose pull, you see an error message like

Traceback (most recent call last):
  File "/usr/bin/docker-compose", line 33, in <module>
    sys.exit(load_entry_point('docker-compose==1.27.4', 'console_scripts', 'docker-compose')())
  File "/usr/lib/python3.8/site-packages/compose/cli/main.py", line 67, in main
    command()
  File "/usr/lib/python3.8/site-packages/compose/cli/main.py", line 123, in perform_command
    project = project_from_options('.', options)
  File "/usr/lib/python3.8/site-packages/compose/cli/command.py", line 60, in project_from_options
    return get_project(
  File "/usr/lib/python3.8/site-packages/compose/cli/command.py", line 131, in get_project
    client = get_client(
  File "/usr/lib/python3.8/site-packages/compose/cli/docker_client.py", line 41, in get_client
    client = docker_client(
  File "/usr/lib/python3.8/site-packages/compose/cli/docker_client.py", line 170, in docker_client
    client = APIClient(**kwargs)
  File "/usr/lib/python3.8/site-packages/docker/api/client.py", line 197, in __init__
    self._version = self._retrieve_server_version()
  File "/usr/lib/python3.8/site-packages/docker/api/client.py", line 221, in _retrieve_server_version
    raise DockerException(
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

Solution:

This means you haven’t started your docker service!

First, try to start it using

sudo systemctl start docker

or

sudo service docker start

or

sudo /etc/init.d/docker restart

(whatever works with your distribution).

After that, retry the command that originally caused the error message to appear.

In case it still shows the same error message, try the following steps:

  • First, check /var/log/docker.log using
    cat /var/log/docker.log

    Check that file for errors during docker startup.

  • Also check if the user you’re running the command as is a member of the docker group. While insufficient permissions will not cause a FileNotFoundError(2, 'No such file or directory')), but a Permission denied, the error message might look similar in some cases.
Posted by Uli Köhler in Container, Docker, Linux

How to fix Synology Docker: failed to initialize logging driver: database is locked

Problem:

When you try to start a specific Docker container using the Synology NAS GUI, the container is being stopped unexpectedly and you see an error message like this in the logs:

Start container mycontainer failed: {"message":"failed to initialize logging driver: database is locked"}.
Signal container mycontainer failed: {"message":"Cannot kill container: mycontainer: Container 5136ddceeb46004c5b18f04eb9ec10cac3808938515874fc31185b0964232201 is not running"}.

Solution:

I fixed this problem by stopping the container, duplicating the container session: Right click on the container -> Settings -> Duplicate Settings

That will create a new container with the given settings. Note that local ports will be set to Auto and will not be copied over, so if you use fixed local ports, you need to set them to a different value in the original container and then set the local ports on the new container to the desired fixed value. Also note that files inside the container are not copied over. In my configuration, all relevant files are stored in mapped volumes on the NAS.

The root cause of this issue seems to be that the logging database for this specific container has been locked by some process. The issue is always limited to a certain container and will not affect other containers (though it could in principle occur for more than one container). I know that at least in my specific case, the issue is not caused by a reboot and will also not be fixed by a reboot of the Synology NAS. Just before I encountered the issue, my NAS had not been rebooted for months, but it might be related to Synology package updates since I updated some packages using the Package manager just before encountering the issue, including a Synology Mail Plus update which failed on the first attempt, but succeeded when I clicked Update again.

Posted by Uli Köhler in Docker, Networking

A modern Kimai setup using docker-compose and nginx

This is the setup I use to run multiple productive kimai instances. In my example, I create the files in /opt/kimai-mydomain. The folder name is not critical, but it is helpful to distinguish multiple indepedent kimai instances.

First, let’s create /opt/kimai-mydomain/docker-compose.yml. You don’t need to modify anything in this file as every relevant configuration is loaded from .env using environment variables.

version: '3.5'
services:
  mariadb:
    image: mariadb:latest
    environment:
      - MYSQL_DATABASE=kimai
      - MYSQL_USER=kimai
      - MYSQL_PASSWORD=${MARIADB_PASSWORD}
      - MYSQL_ROOT_PASSWORD=${MARIADB_ROOT_PASSWORD}
    volumes:
      - ./mariadb_data:/var/lib/mysql
    command: --default-storage-engine innodb
    restart: unless-stopped
    healthcheck:
      test: mysqladmin -p${MARIADB_ROOT_PASSWORD} ping -h localhost
      interval: 20s
      start_period: 10s
      timeout: 10s
      retries: 3

  kimai:
    image: kimai/kimai2:apache-debian-master-prod
    environment:
      - APP_ENV=prod
      - TRUSTED_HOSTS=localhost,${HOSTNAME}
      - [email protected]
      - ADMINPASS=${KIMAI_ADMIN_PASSWORD}
      - DATABASE_URL=mysql://kimai:${MARIADB_PASSWORD}@mariadb/kimai
    volumes:
      - ./kimai_var:/opt/kimai/var
    ports:
      - '17919:8001'
    depends_on:
      - mariadb
    restart: unless-stopped

Now we’ll create the configuration in /opt/kimai-mydomain/.env:

MARIADB_ROOT_PASSWORD=eishi5Pae3chai1Aeth2wiuCh7Ahhi
MARIADB_PASSWORD=su1aesheereithubo0iedootaeRooT
KIMAI_ADMIN_PASSWORD=toiWaeShaiz5Yeifohngu6chunuo6C
[email protected]
HOSTNAME=kimai.mydomain.com

Generate random passwords for .env ! Do NOT leave the default passwords in .env !

You also need to set KIMAI_ADMIN_EMAIL and HOSTNAME correctly.

We can now create the kimai data directory and set the correct permissions:

mkdir -p kimai_var
chown -R 33:33 kimai_var

(33 is the user ID and group ID of the www-data user inside the container)

Now, we will initialize the kimai database and the user:

docker-compose run kimai console kimai:install -n

Once you see a line like

[Sun Mar 07 23:53:35.986477 2021] [core:notice] [pid 50] AH00094: Command line: '/usr/sbin/apache2 -D FOREGROUND'

stop the process using Ctrl+C as this means that Kimai has finished installing.

Now we can create a systemd service that automatically starts Kimai using TechOverflow’s method from Create a systemd service for your docker-compose project in 10 seconds:

curl -fsSL https://techoverflow.net/scripts/create-docker-compose-service.sh | sudo bash /dev/stdin

Now we only need to create an nginx config for reverse proxying of your Kimai domain. There is nothing special to be considered for the config, hence I’ll show my config just as an example that you can copy and paste.

server {
    server_name  kimai.mydomain.com;

    location / {
        proxy_pass http://localhost:17919/;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_redirect default;
    }

    listen [::]:443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/kimai.mydomain.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/kimai.mydomain.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot

}
server {
    if ($host = kimai.mydomain.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

    server_name  kimai.mydomain.com;

    listen [::]:80; # managed by Certbot
    return 404; # managed by Certbot
}

After setting up your config – I always recommend to setup TLS using Let’s Encrypt, even for test setups, open your Browser and go to your Kimai domain, e.g. to https://kimai.mydomain.com. You can directly login to kimai using KIMAI_ADMIN_EMAIL and KIMAI_ADMIN_PASSWORD as specified in .env.

Posted by Uli Köhler in Container, Docker

How to install python3 pip / pip3 in Alpine Linux

Problem:

You want to install pip3 (also called python3-pip) in Alpine linux, but running apk install python3-pip shows you that the package doesn’t exist

/ # apk add python3-pip
ERROR: unable to select packages:
  python3-pip (no such package):
    required by: world[python3-pip]

Solution:

You need to install py3-pip instead using

apk add py3-pip

Example output:

/ # apk add py3-pip
(1/35) Installing libbz2 (1.0.8-r1)
(2/35) Installing expat (2.2.10-r1)
(3/35) Installing libffi (3.3-r2)
[...]

 

Posted by Uli Köhler in Alpine Linux, Container, Docker, Linux

A modern Docker-Compose config for Etherpad using nginx as reverse proxy

This is the configuration I use to run my etherpad installations behind nginx:

docker-compose.yml

version: "3.5"
services:
  etherpad:
    image: etherpad/etherpad:latest
    environment:
      - TITLE=My Etherpad
      - DEFAULT_PAD_TEXT=My Etherpad
      - ADMIN_PASSWORD=${ETHERPAD_ADMIN_PASSWORD}
      - ADMIN_USER=admin
      - DB_TYPE=mysql
      - DB_HOST=mariadb
      - DB_PORT=3306
      - DB_USER=etherpad
      - DB_PASS=${MARIADB_PASSWORD}
      - DB_NAME=etherpad
      - DB_CHARSET=utf8mb4
      - API_KEY=${ETHERPAD_API_KEY}
      - SESSION_REQUIRED=false
    ports:
      - "17201:9001"
    depends_on:
      - mariadb

  mariadb:
    image: mariadb:latest
    environment:
      - MYSQL_ROOT_PASSWORD=${MARIADB_ROOT_PASSWORD}
      - MYSQL_DATABASE=etherpad
      - MYSQL_USER=etherpad
      - MYSQL_PASSWORD=${MARIADB_PASSWORD}
    volumes:
      - './mariadb_data:/var/lib/mysql'
    command: ['mysqld', '--character-set-server=utf8mb4', '--collation-server=utf8mb4_unicode_ci']
    healthcheck:
      test: mysqladmin -p${MARIADB_ROOT_PASSWORD} ping -h localhost
      interval: 20s
      start_period: 10s
      timeout: 10s
      retries: 3

.env

Remember to replace the passwords with your own random passwords !

MARIADB_ROOT_PASSWORD=ue9zahb8Poh1oeMieyaeFaicheecaz
MARIADB_PASSWORD=dieQuoghiu6sao9aiphei7eiquael5
ETHERPAD_API_KEY=een4Chohdiedohzaich0ega5thung6
ETHERPAD_ADMIN_PASSWORD=ahNee3OhR6aiCootaiy7uBui3rooco

nginx config

server {
    server_name  etherpad.mydomain.de;
    access_log off;
    error_log /var/log/nginx/etherpad.mydomain.de-error.log;

    location / {
        proxy_pass http://localhost:17201/;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_redirect default;
    }

    listen [::]:443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/etherpad.mydomain.de/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/etherpad.mydomain.de/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot

}

server {
    if ($host = etherpad.mydomain.de) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    listen [::]:80; # managed by Certbot
    server_name  etherpad.mydomain.de;
    return 404; # managed by Certbot
}

How to autostart

Our post Create a systemd service for your docker-compose project in 10 seconds provides an extremely quick and easy-to-use one-liner to create and autostart a systemd service running docker-compose for your etherpad instance.

How to backup

This will be detailed in a future blogpost.

Posted by Uli Köhler in Container, Docker, nginx

How to use apt install correctly in your Dockerfile

This is the correct way to use apt install in your Dockerfile:

ENV DEBIAN_FRONTEND=noninteractive
RUN apt update && apt install -y PACKAGE && rm -rf /var/lib/apt/lists/*

Key takeaways:

  • Set DEBIAN_FRONTEND=noninteractive to prevent some packages from prompting interactive input (tzdata for example), which leads to indefinite waiting for an user input
  • Run apt update before the install command to fetch the current package lists
  • apt install with -y to prevent apt from asking you if you really want to install the packages
  • rm -rf /var/lib/apt/lists/* after the install command in order ot prevent the cached apt lists (which are fetched by apt update) from ending up in the container image
  • All of that in one command joined by && in order to prevent docker from building separate layers for each part of the command (and to prevent it from first storing /var/lib/apt/lists in one layer and then delete it in another layer)

Also see the official guide on Dockerfile best practices

Posted by Uli Köhler in Container, Docker

How to fix Gitlab CI error during connect: Post http://docker:2375/v1.40/auth: dial tcp: lookup docker on … no such host

Problem:

In your GitLab CI jobs, you see this error message while trying to run some docker command:

error during connect: Post http://docker:2375/v1.40/auth: dial tcp: lookup docker on 192.168.178.1:53: no such host

Solution:

This error occurs with docker-based gitlab runners such as the one we’re that are configured using a docker executor. The error message means that the inner docker container doesn’t have a connection to the host docker daemon.

In order to fix this, set these options in [runners.docker] in your gitlab-runner config.toml:

privileged = true
volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"]

A valid config.toml looks like this:

concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "CoreOS-Haar-Runner"
  url = "https://gitlab.techoverflow.net/"
  token = "bemaiBie8usahMoo9ish"
  executor = "docker"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
    [runners.cache.azure]
  [runners.docker]
    tls_verify = false
    image = "docker:stable"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"]
    shm_size = 0

Why this works

Adding "/var/run/docker.sock:/var/run/docker.sock" allows the inner docker container running inside the gitlab-runner to access the outer (host) docker daemon. Without this, it tries to connect to the docker default URL, i.e. http://docker:2375. This URL can’t be resolved via DNS (the DNS server in my case is 192.168.178.1, the DNS standard port being 53), hence docker prints the error message listed above.

Posted by Uli Köhler in Docker

How to install Kompose on Ubuntu in 10 seconds

Kompose is a tool that converts docker-compose.yml into Kubernetes YAML config files.

You can easily install it globally on most Linux distributions using this command:

sudo curl -L https://github.com/kubernetes/kompose/releases/download/v1.22.0/kompose-linux-amd64 -o /usr/local/bin/kompose
chmod a+x /usr/local/bin/kompose

After that, run

kompose convert

in a directory containing a docker-compose.yml. See the Kompose website for further information.

Posted by Uli Köhler in Container, Docker, Kubernetes, Podman

Minimal Dockerfile example

This is a minimal Dockerfile that you can use to build a test container:

FROM alpine
CMD ["/bin/sh", "-c", "echo 'It works!'"]

Build example:

docker build -t techoverflow-minimal-docker .

Run example:

docker run techoverflow-minimal-docker

Example output:

$ docker run techoverflow-minimal-docker       
It works!

 

Posted by Uli Köhler in Docker

Best-practice configuration for MongoDB with docker-compose

Create /var/lib/mongodb/docker-compose.yml:

version: '3.1'
services:
  mongo:
    image: mongo
    volumes:
        - ./mongodb_data:/data/db
    ports:
        - 27017:27017

This will store the MongoDB data in /var/lib/mongodb/data. I prefer this variant to using docker volumes since this method keeps all MongoDB-related data in the same directory.

Then create a systemd service using

curl -fsSL https://techoverflow.net/scripts/create-docker-compose-service.sh | sudo bash /dev/stdin

See our post on how to Create a systemd service for your docker-compose project in 10 seconds for more details on this method.

You can access MongoDB at localhost:27017! It will autostart after boot

Restart by

sudo systemctl restart mongodb

Stop by

sudo systemctl stop mongodb

View logs:

sudo journalctl -xfu mongodb

View logs in less:

sudo journalctl -xu mongodb

 

Posted by Uli Köhler in Docker, MongoDB

How to disable SELinux in Fedora CoreOS

Warning: Depending on your application, disabling the SELinux security layer might be a bad idea since it might introduce new security risks especially if the container system has security issues.

In order to disable SELinux on Fedora CoreOS, run this:

sudo sed -i -e 's/SELINUX=/SELINUX=disabled #/g' /etc/selinux/config
sudo systemctl reboot

Note that this will reboot your system in order for the changes to take effect.

The sed command will replace the default

SELINUX=enforcing

in /etc/selinux/config to

SELINUX=disabled

 

Posted by Uli Köhler in Container, Docker, Linux

How to export gitlab backup in Gitlab Docker container

GitLab provides an integrated feature to export a TAR file containing all data from the current GitLab instance.

How to make a GitLab backup?

In order to run this for a GitLab instance running on docker-compose use

docker-compose exec gitlab gitlab-backup create STRATEGY=copy

where gitlab is the container running the gitlab image, e.g. gitlab/gitlab-ce:latest.

In case you are running a docker-based setup without docker-compose, run

docker exec my-gitlab-container gitlab-backup create STRATEGY=copy

where my-gitlab-container is the ID or name of the container

Where to find the backups?

By default, gitlab stores the backups in /var/opt/gitlab/backups. In case you need to change this, look for the following line in /etc/gitlab/gitlab.rb:

gitlab_rails['backup_path'] = "/var/opt/gitlab/backups"

In my docker-compose configuration, I map out the entire /var/opt/gitlab directory:

volumes:
   - './data:/var/opt/gitlab'

hence I can find the backups in ./data/backups:

$ ls data/backups/
1607642274_2020_12_10_13.6.3_gitlab_backup.tar
Posted by Uli Köhler in Docker

How to create a systemd backup timer & service in 10 seconds

In our previous post Create a systemd service for your docker-compose project in 10 seconds we introduced a script that automatically creates a systemd service to start a docker-compose-based project. In this post, we’ll show

First, you need to create a file named backup.sh in the directory where docker-compose.yml is located. This file will be run by the systemd service every day. What that file contains is entirely up to you and we will provide examples in future blogposts.

Secondly, run

wget -qO- https://techoverflow.net/scripts/create-backup-service.sh | sudo bash /dev/stdin

from the directory where docker-compose.yml is located. Note that the script will use the directory name as a name for the service and timer that is created. For example, running the script in /var/lib/redmine-mydomain will cause redmine-mydomain-backup to be used a service name.

Example output from the script:

Creating systemd service... /etc/systemd/system/redmine-mydomain-backup.service
Creating systemd timer... /etc/systemd/system/redmine-mydomain-backup.timer
Enabling & starting redmine-mydomain-backup.timer
Created symlink /etc/systemd/system/timers.target.wants/redmine-mydomain-backup.timer → /etc/systemd/system/redmine-mydomain-backup.timer.

The script will create /etc/systemd/systemd/redmine-mydomain-backup.service containing the specification on what exactly to run:

[Unit]
Description=redmine-mydomain-backup

[Service]
Type=oneshot
ExecStart=/bin/bash backup.sh
WorkingDirectory=/var/lib/redmine-mydomain

and /etc/systemd/systemd/redmine-mydomain-backup.timer containing the logic when the .service is started:

[Unit]
Description=redmine-mydomain-backup

[Timer]
OnCalendar=daily
Persistent=true

[Install]
WantedBy=timers.target

and will automatically start and enable the timer. This means: no further steps are needed after running this script!

In order to show the current status of the service, use e.g.

sudo systemctl status redmine-mydomain-backup.timer

Example output:

● redmine-mydomain-backup.timer - redmine-mydomain-backup
     Loaded: loaded (/etc/systemd/system/redmine-mydomain-backup.timer; enabled; vendor preset: enabled)
     Active: active (waiting) since Thu 2020-12-10 02:50:31 CET; 19min ago
    Trigger: Fri 2020-12-11 00:00:00 CET; 20h left
   Triggers: ● redmine-mydomain-backup.service

Dec 10 02:50:31 myserverhostname systemd[1]: Started redmine-mydomain-backup.

In the

Trigger: Fri 2020-12-11 00:00:00 CET; 20h left

line you can see when the service will be run next. By default, the script generates tasks that run OnCalendar=daily, which means the service will be run on 00:00:00 every day. Checkout the systemd.time manpage for further information on the syntax you can use to specify other timeframes.

In order to run the backup immediately (it will still run daily after doing this), do

sudo systemctl start redmine-mydomain-backup.service

(note that you need to run systemctl start on the .service! Running systemctl start on the .timer will only enable the timer and not run the service immediately).

In order to view the logs, use

sudo journalctl -xfu redmine-mydomain-backup.service

(just like above, you need to run journalctl -xfu on the .service, not on the .timer).

In order to disable automatic backups, use e.g.

sudo systemctl disable redmine-mydomain-backup.timer

Source code:

#!/bin/bash
# Create a systemd service & timer that runs the given backup daily
# by Uli Köhler - https://techoverflow.net
# Licensed as CC0 1.0 Universal
export SERVICENAME=$(basename $(pwd))-backup

export SERVICEFILE=/etc/systemd/system/${SERVICENAME}.service
export TIMERFILE=/etc/systemd/system/${SERVICENAME}.timer

echo "Creating systemd service... $SERVICEFILE"
sudo cat >$SERVICEFILE <<EOF
[Unit]
Description=$SERVICENAME

[Service]
Type=oneshot
ExecStart=/bin/bash backup.sh
WorkingDirectory=$(pwd)
EOF

echo "Creating systemd timer... $TIMERFILE"
sudo cat >$TIMERFILE <<EOF
[Unit]
Description=$SERVICENAME

[Timer]
OnCalendar=daily
Persistent=true

[Install]
WantedBy=timers.target
EOF

echo "Enabling & starting $SERVICENAME.timer"
sudo systemctl enable $SERVICENAME.timer
sudo systemctl start $SERVICENAME.timer

 

Posted by Uli Köhler in Docker, Linux

How to use pg_dump in Gitlab Docker container

When using the offical gitlab Docker container, you can use this command to run psql:

docker exec -t -u gitlab-psql [container name] pg_dump -h /var/opt/gitlab/postgresql/ -d gitlabhq_production > gitlab-dump.sql

This will save the SQL dump of the database into gitlab-dump.sql

In case you’re using a docker-compose based setup, use this command:

docker-compose exec -u gitlab-psql gitlab pg_dump -h /var/opt/gitlab/postgresql/ -d gitlabhq_production > gitlab-dump.sql

Note that gitlab in this command is the container name.

Posted by Uli Köhler in Docker, Linux

How to run psql in Gitlab Docker image

When using the offical gitlab Docker container, you can use this command to run psql:

docker exec -t -u gitlab-psql [container name] psql -h /var/opt/gitlab/postgresql/ -d gitlabhq_production

In case you’re using a docker-compose based setup, use this command:

docker-compose exec -u gitlab-psql gitlab psql -h /var/opt/gitlab/postgresql/ -d gitlabhq_production

Note that gitlab in this command is the container name.

Posted by Uli Köhler in Databases, Docker, Linux

How to fix XCP-NG XENAPI_MISSING_PLUGIN(xscontainer) or Error on getting the default coreOS cloud template

Problem:

When creating a CoreOS container on your XCP-NG host, XCP-NG center or XenOrchestra tells you

Cloud config: Error on getting the default coreOS cloud template

with the error message

XENAPI_MISSING_PLUGIN(xscontainer)
This is a XenServer/XCP-ng error

Solution:

Log into the host’s console as root using SSH or the console in XCP-NG center or XenOrchestra and run

yum install xscontainer

After that, reload the page (F5) you use to create your container. No host restart is required.

Note that if you have multiple hosts, you need to yum install xscontainer for each host individually.

Posted by Uli Köhler in Docker, Virtualization

How to backup data from docker-compose MariaDB container using mysqldump

For containers with a MYSQL_ROOT_PASSWORD stored in .env

This is the recommended best practice. For this example, we will assume that .env looks like this:

MARIADB_ROOT_PASSWORD=mophur3roh6eegiL8Eeto7goneeFei

To create a dump:

source .env && docker-compose exec mariadb mysqldump -uroot -p${MARIADB_ROOT_PASSWORD} --all-databases > mariadb-dump-$(date +%F_%H-%M-%S).sql

To restore a dump from mariadb-dump.sql, ensure the container is NOT running before this command:

source .env && docker-compose run -T mariadb mariadb -uroot -p${MARIADB_ROOT_PASSWORD} < mariadb-dump.sql

Note that you have to replace mariadb by the name of your container in docker-compose.yml.

For containers with a MYSQL_ROOT_PASSWORD set to some value not stored in .env

This is secure but you typically have to copy the password multiple times: One time for the mariadb container, one time for whatever container or application uses the database, and one time for any backup script that exports a SQL dump of the entire database

To create a dump:

docker-compose exec mariadb mysqldump -uroot -pYOUR_MARIADB_ROOT_PASSWORD --all-databases > dump-$(date +%F_%H-%M-%S).sql

To restore a dump from mariadb-dump.sql:

docker-compose exec -T mariadb mysql -uroot -pYOUR_MARIADB_ROOT_PASSWORD  < mariadb-dump.sql

Replace YOUR_MARIADB_ROOT_PASSWORD by the password of your installation.

Furthermore, you have to replace mariadb by the name of your container in docker-compose.yml

For containers with MYSQL_ALLOW_EMPTY_PASSWORD=yes

This configuration is a security risk – see The security risk of running docker mariadb/mysql with MYSQL_ALLOW_EMPTY_PASSWORD=yes.

To create a dump:

docker-compose exec mariadb mysqldump -uroot --all-databases > mariadb-dump-$(date +%F_%H-%M-%S).sql

To restore a dump from mariadb-dump.sql:

docker-compose exec -T mariadb mysql -uroot < mariadb-dump.sql

More posts on this topic

TechOverflow is currently planning a post on how to use bup in order to provide quick & efficient backups of docker-based MariaDB/MySQL installations.

Posted by Uli Köhler in Docker