How to delete the baloo index database file

baloo is a KDE desktop search component that indexes files in order to speed up the search.

The index can get quite large, e.g. my index consumes more than 2 GB of HDD space:

$ balooctl indexSize
Actual Size: 2,04 GiB
Expected Size: 1,33 GiB

           PostingDB:     313,32 MiB    22.924 %
         PosistionDB:     521,05 MiB    38.122 %
            DocTerms:     167,93 MiB    12.287 %
    DocFilenameTerms:      53,46 MiB     3.912 %
       DocXattrTerms:            0 B     0.000 %
              IdTree:       9,79 MiB     0.716 %
          IdFileName:      41,71 MiB     3.052 %
             DocTime:      25,80 MiB     1.887 %
             DocData:       2,02 MiB     0.148 %
   ContentIndexingDB:      14,86 MiB     1.087 %
         FailedIdsDB:            0 B     0.000 %
             MTimeDB:       9,12 MiB     0.667 %

If you don’t want to use baloo anyway or if you just want to re-index all files, you might want to delete the entire index:

rm -rf ~/.local/share/baloo

Note that if you haven’t disabled baloo using balooctl stop ; balooctl disable it might silently re-create the index in the background.

Posted by Uli Köhler in Linux

How to fix KaTeX parse error: KaTeX doesn’t work in quirks mode

Problem:

You’re using KaTeX on a website, but every time you try to render, you encounter the following error message:

Uncaught Error: KaTeX parse error: KaTeX doesn't work in quirks mode.
    at new e (VM697 katex.min.js:1)
    at Object.l [as render] (VM697 katex.min.js:1)
[...]

Solution:

KaTeX requires a valid DOCTYPE declaration on your HTML page, so add this line at the top of your HTML file(s), above <html>

<!DOCTYPE html>

Thanks to @xymostech on GitHub for the original solution in the the KaTeX issue tracker:

Posted by Uli Köhler in Javascript, LaTeX

Automated rendering of PDB proteins using PyMol

Downloading the .pdb file

As the RCSB offers direct HTTP acess, this step is trivial.

You can use this shell script to download any protein:

#!/bin/bash
# donwload-pdb.sh
wget http://www.rcsb.org/pdb/files/$1.pdb

Call it with the PDB ID to download, e.g. 1ULI:

./download-pdb.sh 1ULI

Rendering using PyMol

First, install PyMol: Either download it from the website or just use your preferred package manager, e.g.:

sudo apt-get install pymol

By calling PyMol with a script instead of in interactive mode, we can automate the process of rendering an image – manual tuning of the perspective etc is likely to improve the results, however.

The following script integrates both the automatic download and the renderer. A temporary .pml (PyMol script) file is created with static settings

#!/bin/bash
# render-pymol.sh
if [ $# -eq 0 ]
  then
    echo "Usage: render-pymol.sh <PDB ID>"
    exit
fi
#Download
wget -qO $1.pdb http://www.rcsb.org/pdb/files/$1.pdb

#Create the rasmol script
echo "load $1.pdb;" > $1.pml
echo "set ray_opaque_background, on;" >> $1.pml
echo "show cartoon;" >> $1.pml
echo "color purple, ss h;" >> $1.pml
echo "color yellow, ss s;" >> $1.pml
echo "ray 1500,1500;" >> $1.pml
echo "png $1.png;" >> $1.pml
echo "quit;" >> $1.pml

#Execute PyMol
pymol -qxci $1.pml
#Remove temporary files
rm -rf $1.pml $1.pdb

Call it like this:

./render-pymol.sh 1ULI

This command will render the result and store it in 1ULI.png

Customizing the render style

In order to change the size of the generated PNG image, change this line:

echo "ray 1500,1500;" >> $1.pml

The numbers represent the width and height of the generated image. Note that increasing the image size will significantly increase the CPU time required to render the image, especially for complex proteins. Running render-pymol.sh with 1500x1500px to render the 1ULI  took 209 seconds on my Notebook as opposed to 33 seconds for 500×500.

These lines define the style of the rendered protein:

echo "show cartoon;" >> $1.pml
echo "color purple, ss h;" >> $1.pml
echo "color yellow, ss s;" >> $1.pml

while this line set the background color to transparent:

echo "set ray_opaque_background, off;" >> $1.pml

If you prefer a white (non-transparent) background instead, you can add this line right after the line containing load $1.pdb:

echo "bg_color white;" >> $1.pml
echo "set ray_opaque_background, off;" >> $1.pml

Results

1ULI

3ULI

Posted by Uli Köhler in Bioinformatics

How to fix GCC error ‘the lambda has no capture-default’

When encountering a GCC error like this:

error: the lambda has no capture-default

fixing it is usually quite easy. Look for a Lambda function that captures some variable like this

[&myVar] (/* ... */) {/* ... */}

&myVar means “capture myVar by reference”.

In most cases you can just capture all local variables by using a capture default:

[&] (/* ... */) {/* ... */}

In rare cases this will have unintended side-effects as you now are capturing all variables by reference where you might want to capture some by copy – so be sure to check your code.

Note that this error is GCC version-dependent. For me using GCC 7.2 fixed the error.

Posted by Uli Köhler in C/C++, GCC errors

How to fix Angular ‘TypeError: templateRef.createEmbeddedView is not a function’

Problem:

You encounter an error message like this:

ERROR TypeError: templateRef.createEmbeddedView is not a function
    at ViewContainerRef_.createEmbeddedView (core.js:11389)
    at NgIf._updateView (common.js:2843)
    at NgIf.set [as ngIfElse] (common.js:2815)
    at updateProp (core.js:12602)
    at checkAndUpdateDirectiveInline (core.js:12313)
    at checkAndUpdateNodeInline (core.js:13876)
    at checkAndUpdateNode (core.js:13819)
    at debugCheckAndUpdateNode (core.js:14712)
    at debugCheckDirectivesFn (core.js:14653)
    at Object.eval [as updateDirectives] (MyComponent.html:1)

in a component where you have a source code similar to this

<div *ngIf="myCondition ; else elseSection">
    <!-- ... -->
</div>
<div #elseSection>
    <!-- ... -->
</div>

Solution:

Whatever element you reference in the *ngIf else clause can’t be any arbitrary component, but it must be a ng-template.

In order to solve this, change <div #elseSection> to <ng-template #elseSection>. Note that using just <template> is deprecated since Angular4.

The resulting source code should look like this:

<div *ngIf="myCondition ; else elseSection">
    <!-- ... -->
</div>
<ng-template #elseSection>
    <!-- ... -->
</ng-template>
Posted by Uli Köhler in Angular, Javascript

How to fix Angular ‘Cannot find control with unspecified name attribute’

Problem:

In your Angular2/4/5 application you see this error message:

Cannot find control with unspecified name attribute

Solution:

Look for a statement in the HTML angular template like this:

[formControl]="myCtrl"

The error message means that myCtrl can’t be found. Check if this variable is present in your class – it needs be a FormControl which you can import from @angular/forms:

import { FormControl } from '@angular/forms';

In my case, changing it to

[formControl]="myControl"

fixed the issue

Posted by Uli Köhler in Angular, Javascript, Typescript

How to fix Angular4/5/6 ‘No provider for ControlContainer’

Problem:

In your Angular2/4/5 application you’re getting the following error message:

No provider for ControlContainer ("<div class="recall-container mat-elevation-z8">

Solution:

You have not added the @angular/forms FormsModule to your module’s import list.

Go to your app.module.ts and add this line to the imports:

import { FormsModule } from '@angular/forms';

and look for a line like this in your module definition:

imports: [ /* several import modules may be listed here */ ],

and add FormsModule like this (if there are already imports, add FormsModule to the list):

imports: [ FormsModule ],
Posted by Uli Köhler in Angular, Javascript, Typescript

How to fix error TS2339: Property ‘userLanguage’ does not exist on type ‘Navigator’.

Problem:

When developing with typescript,  e.g. with Angular2, you get an error message similar to this one:

error TS2339: Property 'userLanguage' does not exist on type 'Navigator'.

Solution:

Check the error message for the correct file and line. Look for a statement like

window.navigator.userLanguage

Currently typescript does not have userLanguage as a property (tested with typescript up to 2.7.1), although it should already be fixed according to this issue.

You can work around this by simply replacing the statement listed above by

window.navigator['userLanguage']

Using this approach, Typescript will simply not check if the attribute is present or not.

Posted by Uli Köhler in Typescript

Which version on CuDNN should you install for TensorFlow GPU on Ubuntu?

Problem:

You’ve followed my previous blogpost

Fixing TensorFlow libcublas.so.8.0: cannot open shared object file on Ubuntu

on how to install CuBLAS etc. in order to get TensorFlow working.

Now you are getting an error message similar to this:

ImportError: libcudnn.so.6: cannot open shared object file: No such file or directory

You are wondering how you can install CuDNN as it’s not available from your

Solution:

In order to install CuDNN, first go to the NVIDIA CuDNN page. At the time of writing this, downloading CuDNN is only possible if you have an NVIDIA account, so you need to register (click on Join) if you dont have one or Login if you already have one.

On the CuDNN download page you have several versions of CuDNN to choose from. Don’t just download the newest one as TensorFlow requires a specific one.

Look at your error message: It tells you that TensorFlow is missing libcudnn.so.6 – can you see the 6 in that string? That means that you need CuDNN 6.x(TensorFlow 1.5.0, at the time of writing this, always requires CuDNN 6.x). Although you can install CuDNN 7.x, 8.x, 9.x in parallel to 6.x,

Once you have selected the correct version, you need to select a package type.

The first important choice is whether you want a developer package or just the runtime package. You don’t need the developer package to run TensorFlow, even if you are developing applications using TensorFlow! Just select the runtime package.

Regarding the type of package, of course if you are on Linux, you absolutely need to select a linux package. If you use Ubuntu 16.04+, the easiest option is to select cuDNN v6.0 Runtime Library for Ubuntu16.04 (Deb) – even though the name suggest it supports only 16.04, this package worked flawlessly for me on Ubuntu 17.04 and 17.10 as well.

I recommend to download the Ubuntu 16.04 DEB package option unless you have a specific reason not to use it.

Posted by Uli Köhler in GPU, Machine learning

Converting namedtuples to XLSX in Python

This Python snippet allows you to convert an iterable of namedtuple instances to an XLSX file using xlsxwriter.

The header is automatically determined from the first element of the iterable. If the iterable is empty, the resulting XLSX file will also be empty.

import xlsxwriter
import itertools
from collections import namedtuple

def xlsx_write_rows(filename, rows):
    """
    Write XLSX rows from an iterable of rows.
    Each row must be an iterable of writeable values.

    Returns the number of rows written
    """
    workbook = xlsxwriter.Workbook(filename)
    worksheet = workbook.add_worksheet()
    # Write values
    nrows = 0
    for i, row in enumerate(rows):
        for j, val in enumerate(row):
            worksheet.write(i, j, val)
        nrows += 1
    # Cleanup
    workbook.close()
    return nrows


def namedtuples_to_xlsx(filename, values):
    """
    Convert a list or generator of namedtuples to an XLSX file.
    Returns the number of rows written.
    """
    try:
        # Ensure its a generator (next() not allowed on lists)
        values = (v for v in values)
        # Use first row to generate header
        peek = next(values)
        header = list(peek.__class__._fields)
        return xlsx_write_rows(filename, itertools.chain([header], [peek], values))
    except StopIteration:  # Empty generator
        # Write empty xlsx
        return xlsx_write_rows(filename, [])

Example Usage:

MyType = namedtuple("MyType", ["ID", "Name", "Value"])
namedtuples_to_xlsx("test.xlsx", [
    MyType(1, "a", "b"),
    MyType(2, "c", "d"),
    MyType(3, "e", "f"),
])

This example will generate this table:

ID	Name	Value
1	a	b
2	c	d
3	e	f

 

Posted by Uli Köhler in Python

Iterating prime numbers in Python

Problem:

Using Python you want to iterate through consecutive prime numbers

Solution:

You can use gmpy2‘s next_prime() to do this:

import gmpy2

def primes(start=2):
    n = start
    while True:
        n = gmpy2.next_prime(n)
        yield n

# Usage example 1
for prime in primes():
    print(prime)

# Usage example 2
for i, prime in enumerate(primes()):
    print("The {}th prime number is {}".format(i, prime))

 

Posted by Uli Köhler in Mathematics, Python

How to fix GCC error: invalid use of incomplete type ‘class …’

Problem:

You are compiling a C/C++ program using GCC. You get an error message similar to this:

error: invalid use of incomplete type ‘class SomeType’

Solution:

There are multiple possible issues, but in general this error means that GCC can’t find the full declaration of the given class or struct.

The most common issue is that you are missing an #include clause. Find out in which header file the declaration resides, i.e. if the error message mentions class Map, look for something like

class Map {
   // ...
};

Usually the classes reside in header files that are similar to their name, e.g. MyClass might reside in a header file that is called MyClass.h, MyClass.hpp or MyClass.hxx, so be sure to look for those files first. Note that you might also be looking for a type from a library. Often the best approach is to google C++ <insert the missing type here> to find out where it might be located.

Another possible reason is that you have your #include clause after the line where the error occurs. If this is the case, ensure that all required types are included before they are used.

For other reasons, see StackOverflow, e.g. this post

Posted by Uli Köhler in C/C++, GCC errors

Downloading & reading a ZIP file in memory using Python

Problem:

You want to retrieve a ZIP file by downloading it from an URL in Python, but you don’t want to store it in a temporary file and extract it later but instead directly extract its contents in memory.

Solution:

In Python3 can use io.BytesIO together with zipfile (both are present in the standard library) to read it in memory.
The following example function provides a ready-to-use generator based approach on iterating over the files in the ZIP:

import requests
import io
import zipfile

def download_extract_zip(url):
    """
    Download a ZIP file and extract its contents in memory
    yields (filename, file-like object) pairs
    """
    response = requests.get(url)
    with zipfile.ZipFile(io.BytesIO(response.content)) as thezip:
        for zipinfo in thezip.infolist():
            with thezip.open(zipinfo) as thefile:
                yield zipinfo.filename, thefile

 

Posted by Uli Köhler in Allgemein

How to store git HTTPS passwords / credentials permanently so you only have to enter them once

Problem:

Every time you clone a git repository or push/pull, you have to enter a username and a password (e.g. for GitHub or your GitLab installation).
Instead, you want git to store the password so you only have to enter it once.

Solution:

Configure the git credential helper to use a plaintext store instead of the default cache:

git config --global credential.helper store

NOTE: This approach will store your passwords in a plaintext file, so depending on your setup this might be a security risk.

Posted by Uli Köhler in git, Version management

Fixing TensorFlow libcublas.so.8.0: cannot open shared object file on Ubuntu

Problem:

When you run import tensorflow in Python, you get one of the following errors:

ImportError: libcublas.so.8.0: cannot open shared object file: No such file or directory
ImportError: libcusolver.so.8.0: cannot open shared object file: No such file or directory
ImportError: libcudart.so.8.0: cannot open shared object file: No such file or directory
ImportError: libcufft.so.8.0: cannot open shared object file: No such file or directory
ImportError: libcurand.so.8.0: cannot open shared object file: No such file or directory

Solution:

Install the required packages using:

apt-get install libcublas8.0 libcusolver8.0 libcudart8.0 libcufft8.0 libcurand8.0

Note that you also need to install cuDNN – see this followup post

Which version on CuDNN should you install for TensorFlow GPU on Ubuntu?

for details on how to do that.

If this method does not work, you can (as a quick workaround) uninstall tensorflow-gpu and install the tensorflow – the version without GPU support:

pip3 uninstall tensorflow-gpu
pip3 install tensorflow

However, this will likely make your applications much slower.

For other solutions see the TensorFlow bugtracker on GitHub.

Posted by Uli Köhler in Linux, Python

Accessing compilation date and git revision during compilation in HaXe

It is often useful if a program knows the time when it was build and/or the git repository revision id it was build from. Unfortunately, one often forgets to update this information before launching the build. The following code can be used to do this automatically.

(P.S.: The code works even when compiling towards “non-Sys” platforms like JavaScript.)

class BuildInfo {
    /**
     * The time when this library was build.
     */
    public static var buildTime(default, null):Int = getBuildTime();

    /**
     * The git reversion that was used for the build.
     */
    public static var gitRev(default, null):String = getGITRevision();

    /**
     * Macros initalizing the build information.
     */
    public static macro function getBuildTime() {
        var buildTime = Math.floor(Date.now().getTime() / 1000);
        return macro $v{buildTime};
    }
    public static macro function getGITRevision() {
        var gitrev = new sys.io.Process("git", [ "rev-parse", "--verify", "HEAD" ]).stdout.readAll().toString();
        return macro $v{gitrev};
    }

    public static function main() {
        trace("This library was build the " + BuildInfo.buildTime);
        trace("by using revision " + BuildInfo.gitRev);
    }
}
Posted by Yann Spöri in Haxe

The equal, equal operator in HaXe

The “==” operator to check object equality is implemented differently in the various programming languages. For example in Java, the “==” operator checks only the reference and you need the “equals”-Method in order to check the equality of objects:

String s1 = "foo";
String s2 = "foo";
System.out.println("" + (s1 == s2)); // will return false
System.out.println("" + (s1.equals(s2))); // will return true

In other languages like PHP, the “==”-operator can behave even more … strange …:

$a = "3.14159265358979326666666666";
$b = "3.14159265358979323846264338";
echo "" . ($a == $b); // will be 1 == true

So how is the “==”-operator defined in HaXe that can be cross-compiled into multiple languages including PHP and Java?

Well within Haxe, basic types like Int, Float, Strings and Bools are compared by values. So:

class Test {
    static function main() {
        var c1:String = "Hello";
        var c2:String = "World";
        trace(c1 == c2);  // will return false
        var c1:String = "Hello";
        var c2:String = "Hello";
        trace(c1 == c2);  // will return true
    }
}

However nob basic types are compared by-reference:

class C { public function new() {} }

class Test {
    static function main() {
        var c1:C = null;
        var c2:C = null;
        trace(c1 == c2); // will return true
        var c1:C = new C();
        var c2:C = new C();
        trace(c1 == c2); // will return false
    }
}
Posted by Yann Spöri in Haxe

Solving npm Usage of the –dev option is deprecated. Use –only=dev instead.

Problem:

You want to install development dependencies for a NodeJS package using

npm install --dev

but you get this error message:

npm WARN install Usage of the `--dev` option is deprecated. Use `--only=dev` instead.

Solution:

You can use

npm install # Install normal (not development) dependencies
npm install --only=dev # Install only development dependencies

instead. Note that npm install --only=dev will only install development dependencies, so in most cases you want to run both commands.

Posted by Uli Köhler in NodeJS

Using nodemon without a global installation

Problem:

You want to use nodemon in order to automatically reload your NodeJS server, however you don’t want to require a global installation (npm install -g nodemon) but instead install it locally into the node_modules directory:

Solution:

First, install nodemon as dependency (

npm install --save-dev nodemon

We installed it as development dependency for this example, but it will work just as well if you install it as a normal dependency using --save instead of --save-dev.

After that, add a script entry in package.json:

"scripts": {
  "devserver": "./node_modules/nodemon/bin/nodemon.js index.js"
}, /* rest of package.json */

Replace index.js with the name of the file you want to run using nodemon.

Now you can start the development server using

npm run devserver
Posted by Uli Köhler in NodeJS

How to use query string parameters in NodeJS request

Problem:

You’re using the request library in order to make a HTTP GET request:

const request = require("request")

request.get("http://localhost:8000", function(err, response, body) {
    console.log(err, body);
})

Now you’re trying to add query parameters to the request. For this example, we’ll assume that you want to add one parameter: foo=bar

Solution:

You can use the qs parameter like this:

const request = require("request")

request.get({url: "http://localhost:8000", qs: {"foo": "bar"}}, function(err, response, body) {
    console.log(err, body);
})

Note that just adding a qs parameter to request.get won’t work, you need to have a dictionary as first argument that contains at least {"url": <your URL>, "qs": {<one or multiple query parameters>}}

Credits to Daniel at StackOverflow

Posted by Uli Köhler in Javascript