Linux

How to create a partition table using fdisk

Warning: If you run fdisk on the wrong drive here or there is some important data left, you might lose all your data and it will be very hard to restore. Before running these commands, triple-check that you’ve used the correct device (e.g. /dev/sdh)!

In order to create a partition table on a device (e.g. /dev/sdh/dev/sdh1 is not a device but a partition, so using that does not make any sense!), run these commands

sudo fdisk <device file>

If the device doesn’t have a valid partition table, fdisk will automatically create a partition table (but not write it to the disk yet). It will show this output if that is the case (the identifier is random and different every time):

Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0x81ee11ff.

Command (m for help):

If you are sure that you want to run the partition, enter w and press return to write the partition table to disk & exit.

The partition table will be effective immediately, but will not contain any partition. In order to create a partition (for this example we will create one partition being as large as the entire device), run

sudo fdisk <device file>

again.

This time, enter the n command (new partition). When it asks you about the partition type and its size, just press return every time to select the defaults. It should look like this (<return> added to show you where you should press return).

Command (m for help): n
Partition type
p primary (0 primary, 0 extended, 4 free)
e extended (container for logical partitions)
Select (default p): <return>
Partition number (1-4, default 1): <return>
First sector (2048-31143935, default 2048): <return>
Last sector, +sectors or +size{K,M,G,T,P} (2048-31143935, default 31143935): <return>

After that, when fdisk prompts for a command again (i.e. when it says Command (m for help): ), enter w in order to write the changes (i.e. the new partition) to the disk & exit. After fdisk exits, you can see the partition in /dev, e.g. /dev/sdh1

After that, you’ll likely need to create a filesystem on that partition, e.g. sudo mkfs.ext4 /dev/sdh1 or sudo mkfs.vfat /dev/sdh1 . Make sure to create the correct filesystem for the operating system and usecase the device will be used in.

Posted by Uli Köhler in Linux

How to fix APT KEYEXPIRED 1515625755

This error is caused by an expired GPG key that is used in the MongoDB repository.

This key has already been renewed, but you need to tell GPG to update it. Run:

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 0C49F3730359A14518585931BC711F9BA15703C6

For more details, see this GitHub post.

Posted by Uli Köhler in Linux

How to easily find errors in nginx config files

If you edited some nginx config file and nginx doesn’t want to reload or restart, e.g. with an error message like this:

# service nginx reload
Job for nginx.service failed because the control process exited with error code.
See "systemctl  status nginx.service" and "journalctl  -xe" for details.

you likely have some error in one of your config files.

There’s a simple command to check for errors (you need to run it as root): nginx -t

Example output:

nginx: [emerg] unknown directive "autoindex$" in /etc/nginx/sites-enabled/mysite:31
nginx: configuration file /etc/nginx/nginx.conf test failed

Firstly, the last line tells you that there actually is some error in the config files.
The first line tells you exactly where it is: /etc/nginx/sites-enabled/mysite:31 means: Look in the file /etc/nginx/sites-enabled/mysite, line 31.

In this particular case, the actual error message is unknown directive "autoindex$". By checking the aforementioned file I was able to find out that I accidentally entered autoindex $; instead of autoindex on;

After fixing this issue, nginx -t shows that the configuration file seems correct now:

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Note that while most cases of nginx failing to (re)start are caused by issues in the config files, there are some cases in which the config file seems correct and nginx will still not start up. In that case. have a look at the logfile which is commonly located at /var/log/nginx/error.log . You need to be root in order to view it. I recommend this command:

sudo tail -n 1000 /var/log/nginx/error.log
Posted by Uli Köhler in Linux, nginx

How to exit the GNU nano editor?

Just press Ctrl+X. If you dont have unsaved changes, this will exit nano immediately.

In case you have unsaved changes, it will ask you whether to save those changes after pressing.

  • Press Y to tell it to save the changes you’ve made. It will then ask you to check or enter the filename to save to. Once you are finished with the filename, press Enter.
  • Press N to discard all changes (you won’t be able to restore your changes later) and exit nano immediately.
Posted by Uli Köhler in Linux

How to delete the baloo index database file

baloo is a KDE desktop search component that indexes files in order to speed up the search.

The index can get quite large, e.g. my index consumes more than 2 GB of HDD space:

$ balooctl indexSize
Actual Size: 2,04 GiB
Expected Size: 1,33 GiB

           PostingDB:     313,32 MiB    22.924 %
         PosistionDB:     521,05 MiB    38.122 %
            DocTerms:     167,93 MiB    12.287 %
    DocFilenameTerms:      53,46 MiB     3.912 %
       DocXattrTerms:            0 B     0.000 %
              IdTree:       9,79 MiB     0.716 %
          IdFileName:      41,71 MiB     3.052 %
             DocTime:      25,80 MiB     1.887 %
             DocData:       2,02 MiB     0.148 %
   ContentIndexingDB:      14,86 MiB     1.087 %
         FailedIdsDB:            0 B     0.000 %
             MTimeDB:       9,12 MiB     0.667 %

If you don’t want to use baloo anyway or if you just want to re-index all files, you might want to delete the entire index:

rm -rf ~/.local/share/baloo

Note that if you haven’t disabled baloo using balooctl stop ; balooctl disable it might silently re-create the index in the background.

Posted by Uli Köhler in Linux

Fixing TensorFlow libcublas.so.8.0: cannot open shared object file on Ubuntu

Problem:

When you run import tensorflow in Python, you get one of the following errors:

ImportError: libcublas.so.8.0: cannot open shared object file: No such file or directory
ImportError: libcusolver.so.8.0: cannot open shared object file: No such file or directory
ImportError: libcudart.so.8.0: cannot open shared object file: No such file or directory
ImportError: libcufft.so.8.0: cannot open shared object file: No such file or directory
ImportError: libcurand.so.8.0: cannot open shared object file: No such file or directory

Solution:

Install the required packages using:

apt-get install libcublas8.0 libcusolver8.0 libcudart8.0 libcufft8.0 libcurand8.0

Note that you also need to install cuDNN – see this followup post

Which version on CuDNN should you install for TensorFlow GPU on Ubuntu?

for details on how to do that.

If this method does not work, you can (as a quick workaround) uninstall tensorflow-gpu and install the tensorflow – the version without GPU support:

pip3 uninstall tensorflow-gpu
pip3 install tensorflow

However, this will likely make your applications much slower.

For other solutions see the TensorFlow bugtracker on GitHub.

Posted by Uli Köhler in Linux, Python

Fixing LaTeX Error: File … not found on Debian/Ubuntu

Problem:

You’re using latex or pdflatex to compile a .tex file, but you get an error message similar to this one (the solution will work for any missing file, not just utf8x.def):

! LaTeX Error: File `utf8x.def' not found.

Now you’re wondering which package you need to install

Solution 1: Install everything

This problem can often be fixed once and for all by just installing all packages:

sudo apt-get install texlive-full

However, this pulls in a huge amount of packages and is therefore not recommended for most situations.

Solution 2: Install only required package

You can use apt-file to find the package containing the missing file and install it.

First, update the list of files in all known packages (sudo apt-get install apt-file if required):

sudo apt-file update

You only need to do this once every few months or so, before you use apt-file.

Then, look for the missing file (replace utf8x.def by your missing file):

$ apt-file search utf8x.def
texlive-lang-japanese: /usr/share/texlive/texmf-dist/tex/latex/bxbase/bxutf8x.def
texlive-latex-extra: /usr/share/texlive/texmf-dist/tex/latex/ucs/utf8x.def
texlive-luatex: /usr/share/texlive/texmf-dist/tex/lualatex/luainputenc/lutf8x.def

Now it takes some educated guessing which of the three listed packages (texlive-lang-japanese, texlive-latex-extra, texlive-luatex) needs to be installed. In this case, texlive-latex-extrais the correct choice as the other packages list the missing file only in some subdirectory of package (like luainputenc). If in doubt, you can just install all of the listed packages.

Posted by Uli Köhler in LaTeX, Linux

Fixing PPA Unable to identify ‘package’: user@mycomputer in launchpad

Problem:

You’ve uploaded a DEB package to a Launchpad PPA (e.g. using dput), but you get an error message similar to this:

Solution:

You need to use a proper email address (which must be registered in Launchpad) in debian/changelog .

In order to do this, set the $DEBEMAIL environment variable before running dch

Example:

export [email protected]
dch [...]

If $DEBEMAIL is not set, [username]@[hostname] will be used

Posted by Uli Köhler in Linux

Solving Docker permission denied while trying to connect to the Docker daemon socket

Problem:

You are trying to run a docker container or do the docker tutorial, but you only get an error message like this:

docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.26/containers/create: dial unix /var/run/docker.sock: connect: permission denied.
See 'docker run --help'.

Solution:

The error message tells you that your current user can’t access the docker engine, because you’re lacking permissions to access the unix socket to communicate with the engine.

As a temporary solution, you can use sudo to run the failed command as root (e.g. sudo docker ps).
However it is recommended to fix the issue by adding the current user to the docker group:

Run this command in your favourite shell and then completely log out of your account and log back in (or exit your SSH session and reconnect, if in doubt, reboot the computer you are trying to run docker on!):

sudo usermod -a -G docker $USER

After doing that, you should be able to run the command without any issues. Run docker run hello-world as a normal user in order to check if it works. Reboot if the issue still persists.

See What does sudo usermod -a -G group $USER do on Linux? for details on what this command changes on your system and what the parameters mean.

Logging out and logging back in is required because the group change will not have an effect unless your session is closed.

Background information

On Linux, when you run any docker command, the docker binary will try to connect to /var/run/docker.sock. As indicated by its .sock extension, this file is a Unix Domain Socket – basically, a way so multiple processes can communicate on the local computer (also called an IPC mechanism – IPC = “Inter-Process Communication”).

In the case of Docker, the main reason for using the socket is that any user belonging to the docker group can connect to the socket while the Docker daemon itself can run as root. Essentially, it’s a convenience feature and allows multiple docker client commands to communicate to the same daemon process internally.

Posted by Uli Köhler in Container, Docker, Linux

How to interpret smartctl messages like ‘Error: UNC at LBA’?

When running smartctl on your hard drive, you often get a plethora of information that can be hard to interpret for unexperienced users. This post attempts to provide aid in interpreting what the technical reasons behind the error messages are. If you’re looking for advice on whether to replace your hard drive, the only guidance I can give you is it might fail any time, so better backup your data, but it might also run for many years to come.. Furthermore, this article does not describe basic SMART WHEN_FAILED checking but rather interpretation of more subtle signs of possibly impending HDD failures.

Continue reading →

Posted by Uli Köhler in Linux

Automated domain name extraction from Let’s Encrypt certificate transparency logs

A few days ago, Let’s Encrypt into public beta. At the time of writing this article, almost 120k certificateshave been issued, including the certificate for TechOverflow.

I really like the Let’s Encrypt service and I believe it might actually change the way people perceive HTTPS encryption. However, there is one rarely-mentioned side-effect when protecting your domains with their certificates.

Let’s Encrypt publishes certificate transparency logs at crt.sh. This transparency does not come without side-effects, however: crt.sh effectively publishes.

In other words, hiding sites from the public by not publishing their (sub-)domain names anywhere will not work when you issue a certificate for the domain on services like Let’s Encrypt.

Continue reading →

Posted by Uli Köhler in Linux

Fixing ssh: Exited: String too long on OpenWRT

Problem

When trying to execute SSH on OpenWRT with a private key, e.g.

ssh -i id_rsa user@host

you encounter this error:

ssh: Exited: String too long

Continue reading →

Posted by Uli Köhler in Embedded, Linux

nginx Let’s Encrypt authentication for reverse-proxy sites

Problem:

You have an nginx host that is configured as reverse-proxy-only like this:

server {
    server_name  my.domain;
    [...]
    location / {
        proxy_pass http://localhost:1234;
    }
}

For this host, you want to use Let’s Encrypt to automatically issue a certificate using the webroot method like this:

certbot certonly -a webroot --webroot-path ??? -d my.domain

The reverse-proxied webserver does not provide a webroot to use for the automated autentication process and you want to keep the flexibility of updating the cert at any time without manually modifying the nginx configuration.

Continue reading →

Posted by Uli Köhler in Linux, nginx

Solving libhogweed.so.2: undefined symbol: __gmpn_cnd_add_n

After upgrading my server from Debian Wheezy to Jessie, I encountered the following error during apt-get update:

/usr/lib/apt/methods/https: symbol lookup error: /usr/lib/x86_64-linux-gnu/libhogweed.so.2: undefined symbol: __gmpn_cnd_add_n

Continue reading →

Posted by Uli Köhler in Linux

Querying framebuffer resolution in Linux

Problem:

You want to query the current resolution of a screen connected as framebuffer device (e.g. /dev/fb0). Continue reading →

Posted by Uli Köhler in Linux

Fixing bad blocks on HDDs using fixhdd.py

Problem:

You hard drive or SMART tool reports errors when reading specific blocks similar to this message:

[3142.686141] end_request: I/O error, dev sda, sector 31415926

No matter how often you read the block, the hard drive still returns an error and does not reallocate the block.

Continue reading →

Posted by Uli Köhler in Linux, Python

How I solved my Toshiba Linux backlight issues

Symptomatics:

I have both the Toshiba Z830 and R850 for a couple of years now. On both, I’m using the current LTS versions of KUbuntu (at the time of writing this, 14.04). Although, I’m absolutely satisfied with them, there’s a little issue regarding the backlight:

On startup, the backlight works perfectly well. I can change the settings using FN+F6/F7 without any issues. However, after putting the device into standby and waking it up again, pressing said hotkeys shows the backlight percentage dialog, but does not change the brightness.

Because Ubuntu’s SSD reboots are pretty fast Iimply didn’t care about the issue for the past few years. However, out of curiosity, I successfully fixed the issue today.

Continue reading →

Posted by Uli Köhler in Linux

Salt: Increase nginx server_names_hash_bucket_size

Problem:

You use saltstack to automatically deploy configuration to your servers. After installing nginx with the default config, you need to increase the server_names_hash_bucket_size because it won’t startup otherwise.

Continue reading →

Posted by Uli Köhler in Linux, nginx

How to create msgpack DEB packages

Problem:

You want to create a binary DEB package of the msgpack C++ binding. However, there is no official DEB package available.

Continue reading →

Posted by Uli Köhler in Linux

Checking if Hugepages are enabled in Linux

Problem:

On your Linux system, you want to check whether transparent hugepages are enabled on your system.

Solution:

It’s pretty simple:

cat /sys/kernel/mm/transparent_hugepage/enabled

You will get an output like this:

always [madvise] never

You’ll see a list of all possible options ( always, madvise, never ), with the currently active option being enclosed in brackets.madvise is the default.

Continue reading →

Posted by Uli Köhler in Linux, Performance