How to fix ModuleNotFoundError: No module named ‘grpc’ in Python

Problem:

You want to run a Python script that is using some Google Cloud services. However you see an error message similar to this:

[...]
  File "/usr/local/lib/python3.6/dist-packages/google/api_core/gapic_v1/__init__.py", line 16, in <module>
    from google.api_core.gapic_v1 import config
  File "/usr/local/lib/python3.6/dist-packages/google/api_core/gapic_v1/config.py", line 23, in <module>
    import grpc
ModuleNotFoundError: No module named 'grpc'

Solution:

Install the grpcio Python module:

sudo pip3 install grpcio

or, for Python 2.x

sudo pip install grpcio

How to solve permission denied error when trying to echo into a file

Problem:

You are trying to echo a string into a file only accessible to root, e.g.:

sudo echo "foo" > /etc/my.cnf

but you only see this error message:

-bash: /etc/my.cnf: Permission denied

Solution:

Use tee instead (this will also echo the string to stdout)

echo "foo" | sudo tee /etc/my.cfg # Overwrite: Equivalent of echo "foo" > /etc/my.cnf
echo "foo" | sudo tee -a /etc/my.cfg # Append: Equivalent of echo "foo" >> /etc/my.cnf

The reason for the error message is that while echo is executed using sudo, >> /etc/my.cnf is run as normal user (not root).

An alternate approach is to run a sub-shell as sudo:

sudo bash -c 'echo "foo" > /etc/my.cnf'

but this has several caveats e.g. related to escaping so I usually don’t recommend this approach.

How to fix ERROR: Couldn’t connect to Docker daemon at http+docker://localhost – is it running?

Problem:

You want to run a docker-container or docker-compose application, but once you try to start it, you see this error message:

ERROR: Couldn't connect to Docker daemon at http+docker://localhost - is it running?

If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.

Solution:

There are two possible reasons for this error message.

The common reason is that the user you are running the command as does not have the permissions to access docker.

You can fix this either by running the command as root using sudo (since root has the permission to access docker) or adding your user to the docker group:

sudo usermod -a -G docker $USER

and then logging out and logging back in completely (or restarting the system/server).

The other reason is that you have not started docker. On Ubuntu, you can start it using

sudo systemctl enable docker # Auto-start on boot
sudo systemctl start docker # Start right now

 

TechOverflow’s Docker install instructions automatically takes care of starting & enabling the service

How to configure OctoPrint/OctoPi with Ethernet using a static IP

In order to configure OctoPrint/OctoPi to use the Raspberry Pi Ethernet interface with a static IP, first open the rootfs partition on the SD card. After that, open etc/network/interfaces in your preferred text editor (you might need to open it as root, e.g. sudo nano etc/network/interfaces – ensure that you don’t edit your local computer’s /etc/network/interfaces but the one on the SD card).

Now copy the following text to the end of etc/network/interfaces:

auto eth0
allow-hotplug eth0
iface eth0 inet static
    address 192.168.1.234
    netmask 255.255.255.0
    gateway 192.168.1.1
    network 192.168.1.0
    broadcast 192.168.0.255
    dns-nameservers 8.8.8.8 8.8.4.4

You might need to adjust the IP addresses so they match your router.

Save the file and insert the SD card into your Raspberry Pi. You should be able to ping in – in our example, ping 192.168.1.234.

Tested with OctoPrint 0.16.0

Original source: OctoPrint forum

How to fix ‘elasticsearch exited with code 78’

Problem:

You want to run ElasticSearch using docker, but the container immediately stops again using this error message

elasticsearch exited with code 78

or

elasticsearch2 exited with code 78

Solution:

If you look through the entire log message, you’ll find lines like

elasticsearch     | [1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

Therefore we need to increase the vm.max_map_count limit:

sudo sysctl -w vm.max_map_count=524288

Now we need to edit /etc/sysctl.conf so the setting will also be in effect after a reboot.

Look for any vm.max_map_count line in /etc/sysctl.conf. If you find one, set its value to 524288. If there is no such line present, add the line

vm.max_map_count=524288

to the end of /etc/sysctl.conf

Original source: GitHub

 

How to install docker and docker-compose on Ubuntu in 30 seconds

Copy and paste these command blocks into your Linux shell. You need to copy & paste one block at a time – you can paste the next block once the previous block is finished!

# Install prerequisites
sudo apt-get update
sudo apt-get -y install apt-transport-https ca-certificates curl software-properties-common
# Add docker's package signing key
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
# Add repository
sudo add-apt-repository -y "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
# Install latest stable docker stable version
sudo apt-get update
sudo apt-get -y install docker-ce
# Install docker-compose
sudo curl -L "https://github.com/docker/compose/releases/download/1.23.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod a+x /usr/local/bin/docker-compose
# Enable & start docker
sudo systemctl enable docker
sudo systemctl start docker

Note that this will install Docker as deb package whereas docker-compose will be downloaded to /usr/local/bin.

How to install MongoDB CE on Ubuntu in 1 minute

Run these shell commands on your Ubuntu computer to install the current MongoDB community edition and automatically start it (both instantly and on bootup)

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 9DA31620334BD75D9DCB49F368818C72E52529D4
echo "deb [ arch=amd64 ] https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-4.0.list
sudo apt-get update
sudo apt-get install -y mongodb-org
sudo systemctl enable mongod
sudo systemctl start mongod

Source: Official MongoDB documentation

How to run X server using xserver-xorg-video-dummy driver on Ubuntu

When you need to have a dummy display on a headless server (i.e. without any physical monitor attached), first install the dummy driver package:

sudo apt install xserver-xorg-video-dummy

After that, save this config file as dummy-1920x1080.conf

Section "Monitor"
  Identifier "Monitor0"
  HorizSync 28.0-80.0
  VertRefresh 48.0-75.0
  # https://arachnoid.com/modelines/
  # 1920x1080 @ 60.00 Hz (GTF) hsync: 67.08 kHz; pclk: 172.80 MHz
  Modeline "1920x1080_60.00" 172.80 1920 2040 2248 2576 1080 1081 1084 1118 -HSync +Vsync
EndSection

Section "Device"
  Identifier "Card0"
  Driver "dummy"
  VideoRam 256000
EndSection

Section "Screen"
  DefaultDepth 24
  Identifier "Screen0"
  Device "Card0"
  Monitor "Monitor0"
  SubSection "Display"
    Depth 24
    Modes "1920x1080_60.00"
  EndSubSection
EndSection

Now you can start X.org

sudo X -config dummy-1920x1080.conf

The output will look like this:

X.Org X Server 1.19.6
Release Date: 2017-12-20
X Protocol Version 11, Revision 0
Build Operating System: Linux 4.4.0-138-generic x86_64 Ubuntu
Current Operating System: Linux ubuntu-s-1vcpu-2gb-fra1-01 4.15.0-45-generic #48-Ubuntu SMP Tue Jan 29 16:28:13 UTC 2019 x86_64
Kernel command line: BOOT_IMAGE=/boot/vmlinuz-4.15.0-45-generic root=LABEL=cloudimg-rootfs ro console=tty1 console=ttyS0
Build Date: 25 October 2018  04:11:27PM
xorg-server 2:1.19.6-1ubuntu4.2 (For technical support please see http://www.ubuntu.com/support) 
Current version of pixman: 0.34.0
        Before reporting problems, check http://wiki.x.org
        to make sure that you have the latest version.
Markers: (--) probed, (**) from config file, (==) default setting,
        (++) from command line, (!!) notice, (II) informational,
        (WW) warning, (EE) error, (NI) not implemented, (??) unknown.
(==) Log file: "/var/log/Xorg.0.log", Time: Sat Feb 23 17:48:07 2019
(++) Using config file: "dummy-1920x1080.conf"
(==) Using system config directory "/usr/share/X11/xorg.conf.d"

You need to keep this process running as long as you need the display.

Identifying the display number

If no other X server is running, this will, by default, use display number 0. Look for this line to identify the display in use:

(==) Log file: "/var/log/Xorg.0.log", Time: Sat Feb 23 17:48:07 2019

In said line, Xorg.0.log tells you that display 0 is in use, whereas Xorg.1.log tells you that display 1 is in use.

Starting your software

Now you can start your software that needs a graphical interface as follows (we use firefox, and display number 0, as an example):

DISPLAY=:0 firefox

Don’t forget the colon in DISPLAY=:0!

Starting X.org on a specific display number

If you want to start the X server on a specific display number, e.g. 7, because some other dummy server is running concurrently, use this command to start the X server:

sudo X :7 -config dummy-1920x1080.conf

Of course you’ll need to use DISPLAY=:7 firefox in order to start firefox using this config!

In case there is already an X server running on that display, you’ll see an error message like this:

(EE) 
Fatal server error:
(EE) Server is already active for display 7
        If this server is no longer running, remove /tmp/.X7-lock
        and start again.
(EE) 
(EE) 
Please consult the The X.Org Foundation support 
         at http://wiki.x.org
 for help. 
(EE) 

Look for Server is already active for display 7 to be sure you’re seeing the same error message!

How to download a file or directory from a LXC container

To download files, use

lxc file pull <container name>/<path>/<filename> <target directory>

To download directories, use

lxc file pull --recursive <container name>/<path>/<filename> <target directory>

Examples:

Download /root/myfile.txt from mycontainer to the current directory (.):

lxc file pull mycontainer/root/myfile.txt .

Download /root/mydirectory from mycontainer to the current directory (.):

lxc file pull -r mycontainer/root/mydirectory .

 

How to backup Redmine using the Bitnami Docker image

In a previous post I detailed how to install Redmine on Linux using the excellent Bitnami docker image.

This post will teach you how to easily make an online backup of your Redmine installation. Note that automating the backup is not within the scope of this post.

We assume that the redmine is installed as shown in my previous post in /var/lib/redmine. and that you want to backup to my.backup.server:~/redmine-backup/ using rsync.

Backing up the Redmine data

This is pretty easy, as the data is all in just one directory. You can sync it using

rsync --checksum -Pavz /var/lib/redmine/redmine_data my.backup.server:~/redmine-backup/

Note that old versions of files in redmine_data will be overwritten, however files that are deleted locally will not be deleted on the backup server. To me, this seems like a good compromise between the ability to recover deleted files and the used storage space.

Backing up the Redmine database

This part is slightly more complicated, since we need to access the MariaDB server running in a different container. Important note: The container ID can change so it is not sufficient to just find the container ID once and then use it. You need to determine the appropriate ID each time you do a backup. See below on instructions how to do that.

Full command:

docker exec -it $(docker container ls | grep redmine_mariadb_1 | cut -d' ' -f1) mysqldump -uroot bitnami_redmine | xz -e9 -zc - > redmine-database-dump-$(date -I).sql.xz

Let’s break it down:

  • docker exec -it (container ID) (command): Run a command on a running docker container.
  • docker container ls | grep redmine_mariadb_1 | cut -d' ' -f1: Get the ID (first field of the output cut -d' ' -f1) of the running docker container named redmine_mariadb_1
  • mysqldump -uroot bitnami_redmine: This is run on the docker container and dumps the Redmine Database as SQL to stdout. No password is neccessary since the Bitnami MariaDB image allows access without any password.
  • xz -e9 -zc -: Takes the data from mysqldump from stdin (-), compresses it using maximum compression settings (-e9 -z) and writes the compressed data to stdout.
  • > redmine-database-dump-$(date -I).sql.xz: Writes the compressed data from xz into a file called redmine-database-dump-(current date).sql.xz in the current directory.

The resulting file is called e.g. redmine-database-dump-2019-02-01.sql.xz and it’s placed in the current directory. Ensure that you run the command in a suitable directory. Run it in /tmp if you don’t know which directory might be suitable.

Now we can rsync it to the server:

rsync --checksum -Pavz redmine-backup-*.sql.xz my.backup.server:~/redmine-backup/

Since the filename contains the current data, this approach will not overwrite old daily backups of the database, so you can restore your database very flexibly.

How to use custom themes with the Bitnami Redmine Docker image

In a previous post I detailed how to install Redmine on Linux using the excellent Bitnami docker image.

This post shows you how to install a custom theme like A1 (which I used successfully for more than 5 years) if you use the bitnami Docker image. We will assume that you installed redmine in /var/lib/redmine and your systemd service is called redmine.

Note: If you get any permission denied errors, try running the same command using sudo.

First, we need to create the themes directory.

sudo mkdir /var/lib/redmine/themes

The first thing we need to do is to copy the current (default) themes to that directory, since Redmine won’t be able to start up if the default theme isn’t available in the correct version.

In order to do this, we must first ensure that your container is running:

sudo systemctl start redmine

Now we can find out the container ID of the running redmine container:

uli:/var/lib/redmine$ docker container ps | grep redmine
ae4de10d0b41        bitnami/redmine:latest    "/app-entrypoint.sh …"   30 minutes ago      Up 30 minutes   0.0.0.0:3718->3000/tcp   redmine_redmine_1
c231d11c48e9        bitnami/mariadb:latest    "/entrypoint.sh /run…"   30 minutes ago      Up 30 minutes   3306/tcp redmine_mariadb_1

From these lines, you need to select the line that says redmine_redmine_1 at the end. The one that lists redmine_mariadb_1 at the end is the database container and we don’t need that one for this task.

From that line, copy the first column – this is the container ID – e.g. ae4de10d0b41 in this example.

Now we can copy the default theme folder:

docker cp ae4de10d0b41:/opt/bitnami/redmine/public/themes /var/lib/redmine/themes

Now copy your custom theme (e.g. the a1 folder) to /var/lib/redmine/themes.

The next step is to fix the permissions. The bitnami container uses the user with UID 1001, so we need to change the owner to that. Repeat this every time you changed something in the themes directory:

sudo chown -R 1001:1001 /var/lib/redmine/themes

At this point we need to edit the docker-compose config (in /var/lib/redmine/docker-compose.yml) to mount /var/lib/redmine/themes in the correct directory. This is pretty easy: Just add - '/var/lib/redmine-szalata/themes:/opt/bitnami/redmine/public/themes' to the volumes section of the redmine container.

The finished config file will look like this:

version: '2'
services:
  mariadb:
    image: 'bitnami/mariadb:latest'
    environment:
      - ALLOW_EMPTY_PASSWORD=yes
    volumes:
      - '/var/lib/redmine/mariadb_data:/bitnami'
  redmine:
    image: 'bitnami/redmine:latest'
    environment:
      - REDMINE_USERNAME=admin
      - REDMINE_PASSWORD=redmineadmin
      - REDMINE_EMAIL=me@gmail.com
      - SMTP_HOST=smtp.gmail.com
      - SMTP_PORT=25
      - SMTP_USER=me@gmail.com
      - SMTP_PASSWORD=yourGmailPassword
    ports:
      - '3718:3000'
    volumes:
      - '/var/lib/redmine/redmine_data:/bitnami'
      - '/var/lib/redmine-szalata/themes:/opt/bitnami/redmine/public/themes'
    depends_on:
      - mariadb

Now you can restart Redmine:

sudo systemctl restart redmine

and set your new theme by selecting it in Administration -> Settings -> Display.

Fixing ScanaStudio error while loading shared libraries: libftd2xx.so: cannot open shared object file on Ubuntu

Problem:

You want to start IkaLogic ScanaStudio, but you see the following error message:

./ScanaStudio: error while loading shared libraries: libftd2xx.so: cannot open shared object file: No such file or directory

Solution:

You need to download the D2XX drivers from the FTDI D2XX drivers page.

Look for your architecture in the columns (usually x64 (64 bit), this is the same as x86_64) and click the link in the Linux row.

This will download a file named something like libftd2xx-x86_64-1.4.8.gz.

Now you can extract the archive using

tar xvf libftd2xx-*

Now we can copy the shared object file to the IkaLogic ScanaStudio directory:

cp release/build/libftd2xx.so.* ~/Ikalogic/ScanaStudio/libftd2xx.so

Now you can start ScanaStudio:

cd ~/Ikalogic/ScanaStudio
./ScanaStudio

Fixing numpy.distutils.system_info.NotFoundError: No lapack/blas resources found on Ubuntu or Travis

Note: If you are on Windows, you can not install scipy using pip! Follow this guide instead: https://www.scipy.org/install.html. This blog post is only for Linux-based systems!

When building some of my libraries on Travis, I encountered this error during

sudo pip3 install numpy scipy --upgrade
numpy.distutils.system_info.NotFoundError: No lapack/blas resources

Solution

Install lapack and blas:

sudo apt-get -y install liblapack-dev libblas-dev

In most cases you will then get this error message:

error: library dfftpack has Fortran sources but no Fortran compiler found

Fix that by

sudo apt-get install -y gfortran

In Travis, you can do it like this in .travis.yml:

before_install:
    - sudo apt-get -y install liblapack-dev libblas-dev gfortran

How to build rav1e on Ubuntu

This will fetch and build the current git master of rav1e. The build process has been tested on Ubuntu 18.04 with rav1e git revision 49dcaada4.

sudo apt update
sudo apt -y install cargo git perl nasm cmake clang pkg-config
# Fetch
git clone https://github.com/xiph/rav1e.git
mv rav1e rav1e-git
cd rav1e-git
git submodule update --init
# Build libaom-av1
cmake aom_build/aom -DAOM_TARGET_CPU=x86_64 -DCONFIG_AV1_ENCODER=0 -DENABLE_TESTS=0 -DENABLE_DOCS=0 -DCONFIG_LOWBITDEPTH=1
make -j$(nproc)
# Build rav1e
cargo build --release
# Copy to parent directory
cp target/

After the build finishes, the rav1e executable is placed in the directory where you ran those commands. You can delete the rav1e-git folder

Download the resulting binary for Ubuntu 18.04 x86_64 here