Linux

How to install python-config on Alpine Linux

I recommend to use Python 3.x since Python 2.x is deprecated!

Installing python-config for Python 3.x

The python3-dev apk package provides /usr/bin/python3-config, so you’ll have to symlink that to /usr/bin/python-config:

apk update
apk add python3-dev
ln -sf /usr/bin/python3-config /usr/bin/python-config

Installing python-config for Python 2.x

Just install python2-dev, which installs /usr/bin/python-config

apk update
apk add python2-dev

 

 

Posted by Uli Köhler in Alpine Linux

How to run Jupyter Hub (multi-user mode) using systemd

The following script will install Jupyter Hub in single user mode (i.e. only a single Linux user can login to Jupyter Hub using the web interface).

Prerequisites

First install Python & PIP, then NodeJS, then Jupyter Hub, then configurable-http-proxy :

curl -sL https://deb.nodesource.com/setup_16.x | sudo -E bash -
sudo apt-get install -y nodejs python3-pip
sudo pip3 install jupyterhub
sudo npm i -g configurable-http-proxy

Installing the Jupyter Hub systemd service

Run the following script using sudo!

#!/bin/bash
# This script installs and enables/starts the JupyterHub systemd service
export NAME=JupyterHub

# Create service file
cat >/etc/systemd/system/${NAME}.service <<EOF
[Unit]
Description=${NAME}

[Service]
Type=simple
ExecStart=/usr/bin/env jupyterhub --port=11569
User=root
Group=root

Restart=on-failure
RestartSec=5s

[Install]
WantedBy=multi-user.target
EOF

# Enable and start service
systemctl enable --now ${NAME}

This script will install a systemd service named JupyterHub autostart it on boot. It is running on port 11569 by default.

Posted by Uli Köhler in Python, systemd

How to run Jupyter Hub (single user mode) using systemd for autostart

Note: This will only allow a single (preconfigured) user to login to Jupyter lab! See How to run Jupyter Hub (multi-user mode) using systemd on how to deploy Jupyter Hub in multi-user mode using systemd, which allows any Unix user to login!

The following script will install Jupyter Hub in single user mode (i.e. only a single Linux user can login to Jupyter Hub using the web interface).

Prerequisites

First install Python & PIP, then NodeJS, then Jupyter Hub, then configurable-http-proxy :

curl -sL https://deb.nodesource.com/setup_14.x | sudo -E bash -
sudo apt-get install -y nodejs python3-pip
sudo pip3 install jupyterhub
sudo npm i -g configurable-http-proxy

Installing the Jupyter Hub systemd service

Run the following script as the user you want to be able to login! Do not run the script using sudo !

#!/bin/bash
# This script installs and enables/starts a systemd service
export NAME=JupyterHub-$USER
export GROUP=$(id -gn $USER)
# Create service file
sudo tee /etc/systemd/system/${NAME}.service <<EOF
[Unit]
Description=${NAME}

[Service]
Type=simple
ExecStart=/usr/bin/env jupyterhub

WorkingDirectory=$HOME
User=$USER
Group=$GROUP

Restart=on-failure
RestartSec=5s

[Install]
WantedBy=multi-user.target
EOF

# Enable and start service
systemctl enable --now ${NAME}

This script will install a systemd service named JupyterHub-$USER (where $USER is the current user, e.g. uli) and autostart it on boot.

Running multiple services

If you run multiple services, you can run the script for each user and choose a unique port for each service by adding --port=7219 to the /usr/bin/env jupyter hub command, e.g.

ExecStart=/usr/bin/env jupyterhub --port=7192

Alternatively, you can run a single systemwide Jupyter Hub in multi-user mode where multiple users can log in.

Posted by Uli Köhler in Python, systemd

How to run Jupyter Lab as systemd service

If you want to run your Jupyter Lab as a network service on any modern Linux distribution, you can install a systemd service that runs Jupyter. First, you need to install jupyter lab using

sudo pip3 install jupyterlab

In case you don’t have pip3, use sudo apt -y install python3-pip or the equivalent on your distribution.

Note that this script will run Jupyter without token authentication and without password and it will listen on any IP (--ip=0.0.0.0) by default. Either change the command line flags or be aware of the security implications !

#!/bin/bash
# This script installs and enables/starts a systemd service
export NAME=Jupyter

# Create service file
cat >/etc/systemd/system/${NAME}.service <<EOF
[Unit]
Description=${NAME}

[Service]
Type=simple
ExecStart=/usr/bin/env jupyter lab --ip=0.0.0.0 --port 17256 --LabApp.token=''

WorkingDirectory=/home/uli/jupyter
User=uli
Group=uli

Restart=on-failure
RestartSec=5s

[Install]
WantedBy=multi-user.target
EOF

# Enable and start service
systemctl enable --now ${NAME}

You need to change the following entries in the script in order to make it work for you:

WorkingDirectory=/home/uli/jupyter
User=uli
Group=uli

Set the WorkingDirectory to whatever directory you want Jupyter to run in. Note that anyone being able to access the webinterface will basically have full access to that directory!

Set User and Group to the user that should run. Note that running Jupyter as root is not allowed. In case you still want to do it, add the --allow-root flag to the command line options.

Now run the script as root to install the service:

sudo ./install-jupyter-service.sh

Now you can access Jupyter at http://[ip of the computer]:17256.

Changing the configuration

In order to change the configuration, I recommend to edit /etc/systemd/systemd/Jupyter.service (or /etc/systemd/systemd/${NAME}.service if you changed export NAME=Jupyter) directly. After that, run

sudo systemctl daemon-reload
sudo systemctl restart Jupyter

You can also change the installation script and re-run it, but you still need to run daemon-reload and restart.

Running multiple Jupyter instances

In order to run  multiple instances, just run multiple copies of the installation script with different names. For example, use

export NAME=Jupyter-DeepLearning

Debugging Jupyter Lab output

If you have problems with Juypter Lab starting up, use

sudo systemctl status Jupyter

in order to view the status and

sudo journalctl -xfu Jupyter

to view all the logs.

Status output example:

● Jupyter.service - Jupyter
     Loaded: loaded (/etc/systemd/system/Jupyter.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2021-06-11 03:44:28 CEST; 4s ago
   Main PID: 48753 (jupyter-lab)
      Tasks: 1 (limit: 14226)
     Memory: 51.7M
     CGroup: /system.slice/Jupyter.service
             └─48753 /usr/bin/python3 /usr/local/bin/jupyter-lab --ip=0.0.0.0 --port 17256 --LabApp.token=

Jun 11 03:44:29 uli-desktop env[48753]: [I 2021-06-11 03:44:29.215 ServerApp] nbclassic | extension was successfully loaded.
Jun 11 03:44:29 uli-desktop env[48753]: [I 2021-06-11 03:44:29.216 LabApp] JupyterLab extension loaded from /usr/local/lib/python3.8/dist-packages/jupyterlab
Jun 11 03:44:29 uli-desktop env[48753]: [I 2021-06-11 03:44:29.216 LabApp] JupyterLab application directory is /usr/local/share/jupyter/lab
Jun 11 03:44:29 uli-desktop env[48753]: [I 2021-06-11 03:44:29.219 ServerApp] jupyterlab | extension was successfully loaded.
Jun 11 03:44:29 uli-desktop env[48753]: [I 2021-06-11 03:44:29.220 ServerApp] Serving notebooks from local directory: /dev/shm
Jun 11 03:44:29 uli-desktop env[48753]: [I 2021-06-11 03:44:29.220 ServerApp] Jupyter Server 1.8.0 is running at:
Jun 11 03:44:29 uli-desktop env[48753]: [I 2021-06-11 03:44:29.220 ServerApp] http://uli-desktop:17256/lab
Jun 11 03:44:29 uli-desktop env[48753]: [I 2021-06-11 03:44:29.220 ServerApp]     http://127.0.0.1:17256/lab
Jun 11 03:44:29 uli-desktop env[48753]: [I 2021-06-11 03:44:29.220 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
Jun 11 03:44:29 uli-desktop env[48753]: [W 2021-06-11 03:44:29.224 ServerApp] No web browser found: could not locate runnable browser.

Uninstalling the Jupyter Lab service

In order to just stop and disable autostart (but not uninstall) the jupyter lab service, use

sudo systemctl disable --now Jupyter

After that, you can just remove the service file in order to permanently uninstall the service:

sudo rm /etc/systemd/system/Jupyter.service

You can always reinstall using our installation script.

Note that if you have changed the export NAME=... line, you need to replace Jupyter by the value of Name

Posted by Uli Köhler in Python, systemd

Autoinstall script for systemd timer and associated service

The following script will install a systemd .timer file and the associated .service file  into /etc/systemd/system/ and enable the timer (i.e. start on boot) and start the timer immediately.

#!/bin/bash
# This script installs and enables/starts a systemd timer
# It also installs the service file that is run by the given timer
export NAME=MyService

cat >/etc/systemd/system/${NAME}.service <<EOF
# ADD SYSTEMD SERVICE FILE CONTENT HERE
EOF

cat >/etc/systemd/system/${NAME}.timer <<EOF
# ADD SYSTEMD TIMER FILE CONTENT HERE
EOF

# Enable and start service
systemctl enable --now ${NAME}.timer

In order to modify the script for your systemd service, replace MyService by the desired name of your service in

export NAME=MyService

insert the content of your .service file at

# ADD SYSTEMD SERVICE FILE CONTENT HERE

and insert the content of your .timer file at

# ADD SYSTEMD TIMER FILE CONTENT HERE

Full example

The following complete example installs a systemd service named MyService that runs /usr/bin/python3 myscript.py every two hours:

#!/bin/bash
# This script installs and enables/starts a systemd timer
# It also installs the service file that is run by the given timer
export NAME=MyService

cat >/etc/systemd/system/${NAME}.service <<EOF
[Unit]
Description=${NAME}

[Service]
Type=oneshot
ExecStart=/usr/bin/python3 myscript.py
WorkingDirectory=/opt/myservice
EOF

cat >/etc/systemd/system/${NAME}.timer <<EOF
[Unit]
Description=${NAME} timer

[Timer]
OnCalendar=*-*-* 00,02,04,06,08,10,12,14,16,18,20,22:00:00
Persistent=true

[Install]
WantedBy=timers.target
EOF

# Enable and start service
systemctl enable --now ${NAME}.timer
Posted by Uli Köhler in systemd

How to run systemd timer every five minutes

The syntax to run a systemd timer every minute is:

OnCalendar=*-*-* *:00,05,10,15,20,25,30,35,40,45,50,55:00

i.e. run it on the first second (:00) of every 5th minute (00,05,10,15,20,25,30,35,40,45,50,55).

Posted by Uli Köhler in systemd

How to run systemd timer every three hours

The syntax to run a systemd timer every three hours is:

OnCalendar=*-*-* 00,03,06,09,12,15,18,21:00:00

i.e. run it on the first minute and first second (00:00) of every second hour (00,03,06,09,12,15,18,21).

Posted by Uli Köhler in systemd

How to run systemd timer every two hours

The syntax to run a systemd timer every two hours is:

OnCalendar=*-*-* 00,02,04,06,08,10,12,14,16,18,20,22:00:00

i.e. run it on the first minute and first second (00:00) of every second hour (00,02,04,06,08,10,12,14,16,18,20,22).

Posted by Uli Köhler in systemd

How to run systemd timer every minute

The syntax to run a systemd timer every minute is:

OnCalendar=*-*-* *:*:00

i.e. run it on the first second (00) every minute.

Posted by Uli Köhler in systemd

Autoinstall script for systemd service

The following script will install a systemd service file into /etc/systemd/system/ and enable it (i.e. start on boot) and start it immediately.

#!/bin/bash
# This script installs and enables/starts a systemd service
export NAME=MyService

# Create service file
cat >/etc/systemd/system/${NAME}.service <<EOF
# ADD SYSTEMD SERVICE FILE CONTENT HERE
EOF

# Enable and start service
systemctl enable --now ${NAME}

In order to modify the script for your systemd service, replace MyService by the desired name of your service in

export NAME=MyService

and insert the content of your .service file at

# ADD SYSTEMD SERVICE FILE CONTENT HERE

Full example

The following complete example installs a systemd service named MyService that runs /usr/bin/python3 myscript.py:

#!/bin/bash
# This script installs and enables/starts a systemd service
export NAME=MyService

# Create service file
cat >/etc/systemd/system/${NAME}.service <<EOF
[Unit]
Description=MyService

[Service]
Type=oneshot
ExecStart=/usr/bin/python3 myscript.py
WorkingDirectory=/opt/myservice

[Install]
WantedBy=multi-user.target
EOF

# Enable and start service
systemctl enable --now ${NAME}

 

Posted by Uli Köhler in systemd

How to install ruby & rubygems in Alpine Linux

Problem:

You want to install ruby and the gem package manager in Alpine linux, but running apk install ruby rubygems shows you that the package doesn’t exist

/ # apk add ruby rubygems
ERROR: unable to select packages:
  rubygems (no such package):
    required by: world[rubygems]

Solution:

gem is included in the ruby package. So the only command you need to run is

apk update
apk add ruby

Example output:

/ # apk add ruby
(1/7) Installing ca-certificates (20191127-r5)
(2/7) Installing gdbm (1.19-r0)
(3/7) Installing gmp (6.2.1-r0)
(4/7) Installing readline (8.1.0-r0)
(5/7) Installing yaml (0.2.5-r0)
(6/7) Installing ruby-libs (2.7.3-r0)
(7/7) Installing ruby (2.7.3-r0)
Executing busybox-1.32.1-r6.trigger
Executing ca-certificates-20191127-r5.trigger
OK: 928 MiB in 154 packages

After doing that, you can immediately use both ruby and gem.

Posted by Uli Köhler in Alpine Linux, Container, Docker, Linux, Ruby

How to enable SSH root login on Alpine Linux

Note: Be aware that enabling root access via SSH has security implications!

On Alpine Linux, root SSH access using passwords is disabled by default. The following tutorial shows you how to enable password-based root login via SSH when using openssh. (I have not tested whether root access is enabled when installing Alpine Linux using dropbear instead of openssh)

First, open the SSH config file using

vi /etc/ssh/sshd_config

Now look for this line:

#PermitRootLogin prohibit-password

Press I in order to activate vi editing mode.

Remove the # at the beginning of the line and change prohibit-password to yes:

PermitRootLogin yes

Now save and exit by pressing Esc and then pressing :wq and Enter.

After that, restart openssh using

service sshd restart

Now you can login as root using the password.

Posted by Uli Köhler in Alpine Linux

How to change keyboard layout in Alpine Linux

You can easily change the keyboard layout in Alpine Linux by running

setup-keymap

This will first prompt you for your generic keyboard layout:

Available keyboard layouts:
af     ara    ba     bg     by     cm     de     ee     fi     gb     gr     id     in     is     ke     kz     lk     ma     mk     mt     nl     pk     ro     se     sy     tm     ua     vn
al     at     bd     br     ca     cn     dk     epo    fo     ge     hr     ie     iq     it     kg     la     lt     md     ml     my     no     pl     rs     si     th     tr     us
am     az     be     brai   ch     cz     dz     es     fr     gh     hu     il     ir     jp     kr     latam  lv     me     mm     ng     ph     pt     ru     sk     tj     tw     uz
Select keyboard layout: [none]

In my case, I just type de to select the German keyboard layout and press Enter.

After that, It will prompt you for the keyboard variant to use. For German keyboards this will look like this:

Available variants: de-T3 de-deadacute de-deadgraveacute de-deadtilde de-dsb de-dsb_qwertz de-dvorak de-e1 de-e2 de-mac de-mac_nodeadkeys de-neo de-nodeadkeys de-qwerty de-ro de-ro_nodeadkeys de-ru de-sundeadkeys de-tr de-us de
Select variant (or 'abort'):

If you don’t particularly care about the variant or if you don’t know what than means, just enter whatever you entered in the first step: If you entered de before, enter de again and press Enter.

Now Alpine will configure your Keyboard:

* WARNING: you are stopping a boot service
* Caching service dependencies ... [ ok ]
* Setting keymap ... [ ok ]

The new keymap will be effective immediately and it will also persist after a reboot.

Note that the Alpine Installer (which you can start using setup-alpine) will automatically ask you for the correct keymap – hence, it’s not neccessary to run setup-keymap before running the installer.

Posted by Uli Köhler in Alpine Linux

How to run mkpasswd with yescrypt on Ubuntu/Debian

Currently the Ubuntu/Debian mkpasswd command does not support yescrypt.

In order to use it anyway, we can use the ulikoehler/mkpasswd docker image to run the proper version of mkpasswd:

docker run --rm -it ulikoehler/mkpasswd

This will prompt you for a password and then echo the yescrypt encrypted and salted password:

$ docker run --rm -it ulikoehler/mkpasswd
Password:
$y$j9T$YzrfO5lQkDWahpz5pwYzg/$HzQoMYt.7E1jj.sd6OyYCGI/Qk6oGehNgz5uvY1qp59

 

Posted by Uli Köhler in Docker, Linux

How to autostart service from /etc/init.d on Alpine Linux

In order to autostart a service on Alpine Linux, use

rc-update add [service] default

For example, autostart docker using

rc-update add docker default

 

Posted by Uli Köhler in Alpine Linux

How to install XCP-NG Guest Utilities on Alpine Linux

At the time of writing this, the XCP-NG guest tools ISO does not support Alpine Linux, but you can install the guest utilities using

apk add xe-guest-utilities

Additionally, you need to enable starting the Xen guest utilities on startup:

rc-update add xe-guest-utilities default

Now we can start the utilities manually in order to avoid having to reboot:

/etc/init.d/xe-guest-utilities start
Posted by Uli Köhler in Alpine Linux

How to fix Alpine Linux ERROR: ‘install’ is not an apk command

Problem:

You are trying to install an apk package using e.g.

apk install python

but you see this error message:

ERROR: 'install' is not an apk command. See 'apk --help'.

Solution:

Use apk add instead of apk install to install packages on Alpine Linux !

So instead of

apk install python

use

apk add python

 

Posted by Uli Köhler in Alpine Linux

How to fix Alpine Linux -ash: sudo: not found

Many users who are used to classical Linux distributions will see the following error message when using sudo on Alpine Linux:

-ash: sudo: not found

You don’t need sudo in many use cases!

The easiest way to fix that is to just run the command as root.

In order to get into a root shell (if you are not already logged in as root), use

su

which expects you to enter the root password.

Now just run the original command without sudo.

Still want to install sudo?

If you still want to install sudo, just run

apk add sudo

This is often the best approach if you have e.g. scripts that are running sudo commands.

Posted by Uli Köhler in Alpine Linux

How to fix docker.errors.DockerException: Error while fetching server API version: (‘Connection aborted.’, FileNotFoundError(2, ‘No such file or directory’))

Problem:

While running a docker command like docker-compose pull, you see an error message like

Traceback (most recent call last):
  File "/usr/bin/docker-compose", line 33, in <module>
    sys.exit(load_entry_point('docker-compose==1.27.4', 'console_scripts', 'docker-compose')())
  File "/usr/lib/python3.8/site-packages/compose/cli/main.py", line 67, in main
    command()
  File "/usr/lib/python3.8/site-packages/compose/cli/main.py", line 123, in perform_command
    project = project_from_options('.', options)
  File "/usr/lib/python3.8/site-packages/compose/cli/command.py", line 60, in project_from_options
    return get_project(
  File "/usr/lib/python3.8/site-packages/compose/cli/command.py", line 131, in get_project
    client = get_client(
  File "/usr/lib/python3.8/site-packages/compose/cli/docker_client.py", line 41, in get_client
    client = docker_client(
  File "/usr/lib/python3.8/site-packages/compose/cli/docker_client.py", line 170, in docker_client
    client = APIClient(**kwargs)
  File "/usr/lib/python3.8/site-packages/docker/api/client.py", line 197, in __init__
    self._version = self._retrieve_server_version()
  File "/usr/lib/python3.8/site-packages/docker/api/client.py", line 221, in _retrieve_server_version
    raise DockerException(
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

Solution:

This means you haven’t started your docker service!

First, try to start it using

sudo systemctl start docker

or

sudo service docker start

or

sudo /etc/init.d/docker restart

(whatever works with your distribution).

After that, retry the command that originally caused the error message to appear.

In case it still shows the same error message, try the following steps:

  • First, check /var/log/docker.log using
    cat /var/log/docker.log

    Check that file for errors during docker startup.

  • Also check if the user you’re running the command as is a member of the docker group. While insufficient permissions will not cause a FileNotFoundError(2, 'No such file or directory')), but a Permission denied, the error message might look similar in some cases.
Posted by Uli Köhler in Container, Docker, Linux

How to install docker & docker-compose on Alpine Linux

In this post, we’ll show how to install docker-compose (and the docker backend) on Alpine Linux.

Step 1: Enable the apk community repository

Edit the repository config as root using

vi /etc/apk/repositories

It should look like this (the exact URLs will vary according to your selected mirror)

#/media/cdrom/apks
http://ftp.halifax.rwth-aachen.de/alpine/v3.13/main
#http://ftp.halifax.rwth-aachen.de/alpine/v3.13/community
#http://ftp.halifax.rwth-aachen.de/alpine/edge/main
#http://ftp.halifax.rwth-aachen.de/alpine/edge/community
#http://ftp.halifax.rwth-aachen.de/alpine/edge/testing

Now find the line that ends in /communityand does NOT end with /edge/communityTypically, this is the second line in the file:

#http://ftp.halifax.rwth-aachen.de/alpine/v3.13/community

Navigate to this line by using the arrow keys, press the Insert key in order to activate Editing.

Then remove the # at the beginning of the line. The resulting file should look like this:

#/media/cdrom/apks
http://ftp.halifax.rwth-aachen.de/alpine/v3.13/main
http://ftp.halifax.rwth-aachen.de/alpine/v3.13/community
#http://ftp.halifax.rwth-aachen.de/alpine/edge/main
#http://ftp.halifax.rwth-aachen.de/alpine/edge/community
#http://ftp.halifax.rwth-aachen.de/alpine/edge/testing

Now save the file an exit the editor by pressing the Esc key, entering :wq and pressing enter.

Step 2: Install docker & docker-compose

Now we’ll fetch the package lists using

apk update

After that, we can install docker and docker-compose using

apk add docker docker-compose

Step 3: Enable docker daemon autostart & start docker daemon

Enable autostart on boot using

rc-update add docker default

and then start docker using

/etc/init.d/docker start
Posted by Uli Köhler in Alpine Linux