Linux

How to free space by cleaning up old systemd / journald logs

If you need more free hard drive space on your system, you can clear old logs

Important note: When you’ve deleted old logs, they are deleted and there is no way to view them any more! Ensure that you don’t need them any more

View size occupied by those logs:

sudo du -sh /var/log/journal/

Tell journald to always only keep the last 25 Megabytes of logs (plus the current, not-rotated-yet logs):

sudo journalctl --vacuum-size=25M

Move current logs to archive (“rotate“) and only keep the last 25 Megabytes of logs:

sudo journalctl --flush --rotate
sudo journalctl --vacuum-size=25M

Alternatively you can tell it to keep only the last hour of logs (plus the current, not-rotated-yet logs):

sudo journalctl --vacuum-time=1h
Posted by Uli Köhler in Linux

How to fix mount: unknown filesystem type ‘smbfs’

Problem:

When you’re trying to mount a Windows network share using a command like

sudo mount -t smbfs //Asus/store_n_go /mnt/

you see this error message:

mount: unknown filesystem type 'smbfs'

Solution:

First ensure samba is installed

sudo apt install samba

then try again using cifs as filesystem type instead of smbfs:

sudo mount -t cifs //Asus/store_n_go /mnt/

 

Posted by Uli Köhler in Linux, Networking

How to print ISO8601 date using ‘date’ on command line

Use one of these commands to print the ISO8601 time using the date tool included in most Linux distributions.

date -I # 2019-04-06
date -Iseconds # 2019-04-06T17:22:49+02:00
date -Ins # 2019-04-06T17:23:08,505995625+02:00

For reference see the date manpage.

Posted by Uli Köhler in Linux

How to install curl on Ubuntu Linux

On most Ubuntu installations, curl is already installed. In order to check, type

curl

into your shell and press return.

If you see a message like

curl: try 'curl --help' or 'curl --manual' for more information

curl is already installed and you don’t need to do anything.

In case you see this message:

Command 'curl' not found, but can be installed with:

apt install curl
Please ask your administrator.

or (more rarely) this message:

-bash: /usr/bin/curl: No such file or directory

you can install curl by copying

sudo apt install curl

into your shell and pressing return.

Posted by Uli Köhler in Linux

How to upload your Python package to PyPI in 30 seconds

Prerequisite: Install twine:

sudo pip3 install twine

Before the next step, ensure you have no uncommitted files, because those will be deleted!

Also, ensure that your package is ready for release. Ensure that you have the correct version listed in setup.py

sudo chown -R $USER: .
git clean -xdf
python3 setup.py sdist
twine upload dist/*

Variant if you don’t have python3:

sudo chown -R $USER: .
git clean -xdf
python setup.py sdist
twine upload dist/*

For more detailed instructions see this post.

Detailed explanation of the commands:

  • sudo chown -R $USER: . Fix permission issues possibly introduced by sudo python3 setup.py install
  • git clean -xdf Remove uncommitted files and other fuzz
  • python3 setup.py sdist Build source package
  • twine upload dist/* This will ask you for user PyPI username and password and then upload the package.
Posted by Uli Köhler in Linux, Python

How to backup all indices from ElasticSearch

You can use elasticdump to backup all indices from your ElasticSearch cluster. Install using

sudo npm install elasticdump -g

If you don’t have npm, see How to install NodeJS 10.x on Ubuntu in 1 minute.

This package installs two binarys: elasticdump (used to dump a single index) and multielasticdump (used to dump multiple indices in parallel)

We can use multielasticdump to dump all indexes:

mkdir -p es_backup
multielasticdump --direction=dump --input=http://localhost:9200 --output=es_backup

Restore using:

multielasticdump --direction=load --input=es_backup --output=http://localhost:9200

 

Posted by Uli Köhler in ElasticSearch, Linux

How to fix OpenVPN ‘failed to find GID for group openvpn’

Problem:

In your OpenVPN server logs you see this error message

failed to find GID for group openvpn

followed by a server restart (Exiting due to fatal error).

Solution:

Run this command to add the OpenVPN group:

sudo groupadd openvpn

In most cases, you’ll see this in your server log after doing that:

failed to find UID for user openvpn
Exiting due to fatal error

In that case, refer to our previous post on How to fix OpenVPN “failed to find UID for user openvpn”

Posted by Uli Köhler in Cryptography, Linux

How to fix OpenVPN ‘failed to find UID for user openvpn’

Problem:

In your OpenVPN server logs you see this error message

failed to find UID for user openvpn

followed by a server restart (Exiting due to fatal error).

Solution:

Run this command to add the openvpn user and add that user to the openvpn group:

sudo useradd openvpn -g openvpn
Posted by Uli Köhler in Cryptography, Linux

How to generate Diffie-Hellman (DH) parameters using OpenSSL

Problem:

For our webserver or VPN server, you want to use unique Diffie-Hellman parameters but you don’t know how to generate the .pem file using OpenSSL.

Solution:

Use this command to generate the parameters and save them in dhparams.pem:

openssl dhparam -out dhparams.pem 4096

This command generates Diffie-Hellman parameters with 4096 bits. This provides good security while still providing a very reasonable performance for modern devices. Depending on your preferred level of Paranoia you might want to increase the number of bits even more.

Note that even for “only” 4096 bits generating the parameters will usually take a couple of minutes. Larger parameter sizes might take many hours to days to generate. Ensure that you are generating the parameters on a fast computer and not on your Raspberry Pi or similar!

Posted by Uli Köhler in Cryptography, Linux

How to enable SSH on Raspbian without a screen

You can open the boot partition on the SD card (the FAT32 partition) and create an empty file named ssh in the root directory of that partition. Ensure that the file is names ssh and not ssh.txt !

If you are in the correct working directory in the command line, use

touch ssh

On recent Ubuntu version, this will switch to the correct directory and create the file (but you need to mount the directory manually e.g. using your file explorer:

cd /media/$USER/boot && touch ssh

Don’t forget to unmount the boot drive before removing the SD card. Once you restart the Raspberry Pi with the modified SD card, SSH will be enabled without you having to attach a keyoard or screen to the Pi.

This approach was tested with the 2018-11-13 version of Raspbian and works with Raspberry Pi 1, Raspberry Pi 2 and Raspberry Pi 3.

Credits to Yahor for the original solution on StackOverflow!

Posted by Uli Köhler in Embedded, Linux

Fixing gcloud WARNING: `docker-credential-gcloud` not in system PATH

Problem:

You want to configure docker to be able to access Google Container Registry using

gcloud auth configure-docker

but you see this warning message:

WARNING: `docker-credential-gcloud` not in system PATH.
gcloud's Docker credential helper can be configured but it will not work until this is corrected.
gcloud credential helpers already registered correctly.

Solution:

Install docker-credential-gcloud using

sudo gcloud components install docker-credential-gcr

In case you see this error message:

ERROR: (gcloud.components.install) You cannot perform this action because this Cloud SDK installation is managed by an external package manager.
Please consider using a separate installation of the Cloud SDK created through the default mechanism described at: https://cloud.google.com/sdk/

use this alternate installation command instead (this command is for Linux, see the official documentation for other operating systems):

VERSION=1.5.0
OS=linux
ARCH=amd64

curl -fsSL "https://github.com/GoogleCloudPlatform/docker-credential-gcr/releases/download/v${VERSION}/docker-credential-gcr_${OS}_${ARCH}-${VERSION}.tar.gz" \
  | tar xz --to-stdout ./docker-credential-gcr \
  | sudo tee /usr/bin/docker-credential-gcr > /dev/null && sudo chmod +x /usr/bin/docker-credential-gcr

After that, configure docker using

docker-credential-gcr configure-docker

Now you can retry running your original command.

For reference, see the official documentation.

Posted by Uli Köhler in Cloud, Container, Docker, Linux

How to install kubectl on Ubuntu

Problem:

You want to run the Kubernetes kubectl command on Ubuntu but you see an error message like this:

command not found: kubectl

Solution:

Install kubectl using snap:

sudo snap install kubectl --classic

After this command has finished installing kubectl, in most cases you can use it immediately. In case you still get the command not found: kubectl error message, run $SHELL to reload your shell and check if /snap/bin is in your $PATH environment variable.

Posted by Uli Köhler in Container, Kubernetes, Linux

How to fix Linux/Windows dual boot clock shift

Problem:

You have a dual-boot system. Every time you reboot from Linux to Windows, the time is shifted by several hours.

Solution:

On Linux, run

sudo timedatectl set-local-rtc 1

This will configure Linux to store local time in the RTC.

See this StackOverflow post for alternate solutions

Background:

Both Linux and Windows use the hardware clock (RTC – Real time clock) integrated into the computer hardware. However, Windows assumes that the RTC stores local time by default whereas Linux assumes the RTC stores UTC time.

Posted by Uli Köhler in Linux, Windows

How to fix ModuleNotFoundError: No module named ‘grpc’ in Python

Problem:

You want to run a Python script that is using some Google Cloud services. However you see an error message similar to this:

[...]
  File "/usr/local/lib/python3.6/dist-packages/google/api_core/gapic_v1/__init__.py", line 16, in <module>
    from google.api_core.gapic_v1 import config
  File "/usr/local/lib/python3.6/dist-packages/google/api_core/gapic_v1/config.py", line 23, in <module>
    import grpc
ModuleNotFoundError: No module named 'grpc'

Solution:

Install the grpcio Python module:

sudo pip3 install grpcio

or, for Python 2.x

sudo pip install grpcio
Posted by Uli Köhler in Cloud, Linux, Python

How to fix echo printing literal \n (backslash newline)

Problem:

You want to echo into a file like this:

echo "\ntest\n" > test.txt

but after doing that, test.txt contains the literal

\ntest\n

Solution:

Use echo -e (-e means: interpret backslash escapes):

echo -e "\ntest\n" > test.txt

After doing that, test.txt will contain test with a newline before and after.

Posted by Uli Köhler in Linux

How to solve permission denied error when trying to echo into a file

Problem:

You are trying to echo a string into a file only accessible to root, e.g.:

sudo echo "foo" > /etc/my.cnf

but you only see this error message:

-bash: /etc/my.cnf: Permission denied

Solution:

Use tee instead (this will also echo the string to stdout)

echo "foo" | sudo tee /etc/my.cfg # Overwrite: Equivalent of echo "foo" > /etc/my.cnf
echo "foo" | sudo tee -a /etc/my.cfg # Append: Equivalent of echo "foo" >> /etc/my.cnf

The reason for the error message is that while echo is executed using sudo, >> /etc/my.cnf is run as normal user (not root).

An alternate approach is to run a sub-shell as sudo:

sudo bash -c 'echo "foo" > /etc/my.cnf'

but this has several caveats e.g. related to escaping so I usually don’t recommend this approach.

Posted by Uli Köhler in Linux

How to fix ERROR: Couldn’t connect to Docker daemon at http+docker://localhost – is it running?

Problem:

You want to run a docker-container or docker-compose application, but once you try to start it, you see this error message:

ERROR: Couldn't connect to Docker daemon at http+docker://localhost - is it running?

If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.

Solution:

There are two possible reasons for this error message.

The common reason is that the user you are running the command as does not have the permissions to access docker.

You can fix this either by running the command as root using sudo (since root has the permission to access docker) or adding your user to the docker group:

sudo usermod -a -G docker $USER

and then logging out and logging back in completely (or restarting the system/server).

The other reason is that you have not started docker. On Ubuntu, you can start it using

sudo systemctl enable docker # Auto-start on boot
sudo systemctl start docker # Start right now

 

TechOverflow’s Docker install instructions automatically takes care of starting & enabling the service

Posted by Uli Köhler in Container, Docker, Linux