Minimal Koa.JS example with Router & Body parser

This is the minimal Koa.JS application which I use as a template for new NodeJS webserver applications.

#!/usr/bin/env node
const router = require('koa-router')();
const koaBody = require('koa-body');
const Koa = require('koa');
const app = new Koa();

app.use(koaBody());

router.get('/', async ctx => {
    ctx.body = "Hello world";
});

app.use(router.routes());

if (!module.parent) app.listen(3000);

Install the requirements using

npm i --save koa koa-router koa-body

and run using

node index.js

assuming you have saved our code from above in index.js.

Now (with the node index.js command still running) go to http://localhost:3000 . You should see Hello world there. Now it’s your turn to continue on your endeavour to develop the world’s greatest webservers 🙂

Posted by Uli Köhler in Javascript, NodeJS

What does ‘if (!module.parent)’ mean in NodeJS?

In NodeJS applications you often see code like

if (!module.parent) {
    app.listen(3000);
}

This means: Run app.listen(3000) only if you are running the file

Suppose this code is in index.js. In this case, the code will only be executed if you run index.js directly (i.e. using node index.js) and not if index.js is required from another file (by require('./index.js');).

If index.js is required from another Javascript module (i.e. file), module.parent will be set to that module.

Posted by Uli Köhler in Javascript, NodeJS

How to fix ‘docker: invalid reference format.’

Problem:

You want to start docker container but you see an error message like this:

docker: invalid reference format.

Solution:

Docker is telling you that the syntax of the docker image name (& version) is wrong. Note that this is not the same as docker not being able to find the image in the registry. Docker will not even be able to lookup the image in the registry if you see an invalid reference format error!

Common causes include:

  • You used a colon at the end of the image name, e.g. ubuntu: – omit the colon ; using just ubuntu will refer to ubuntu:latest
  • You used a dash at the end of the image name, e.g. ubuntu- – omit the dash ; using just ubuntu will refer to ubuntu:latest
  • You used variables like ubuntu:$VERSION but $VERSION is not set. Ensure you have set $VERSION to an appropriate (non-empty) value like latest or 18.04!
  • You used multiple colons like ubuntu:18.04:2 or ubuntu:ubuntu:18.04. Use only a single colon in the image name!
  • You mixed up the order of command line arguments, so another argument is being interpreted as the image name. Check the syntax of your docker command, e.g. docker run and compare some examples with your command.
Posted by Uli Köhler in Container, Docker

How to get schema of SQLite3 table in Python

Also see How to show table schema for SQLite3 table on the command line

Use this function to find the table schema of a SQLite3 table in Python:

def sqlite_table_schema(conn, name):
    """Return a string representing the table's CREATE"""
    cursor = conn.execute("SELECT sql FROM sqlite_master WHERE name=?;", [name])
    sql = cursor.fetchone()[0]
    cursor.close()
    return sql

Usage example:

print(sqlite_table_schema(conn, 'commands'))

Full example:

#!/usr/bin/env python3
import sqlite3
conn = sqlite3.connect('/usr/share/command-not-found/commands.db')

def sqlite_table_schema(conn, name):
    cursor = conn.execute("SELECT sql FROM sqlite_master WHERE name=?;", [name])
    sql = cursor.fetchone()[0]
    cursor.close()
    return sql

print(sqlite_table_schema(conn, 'commands'))

which prints

CREATE TABLE "commands" 
           (
            [cmdID] INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
            [pkgID] INTEGER NOT NULL,
            [command] TEXT,
            FOREIGN KEY ([pkgID]) REFERENCES "pkgs" ([pkgID])
           )

 

 

Posted by Uli Köhler in Python, SQLite

How to fix SQLite3 Python ‘Incorrect number of bindings supplied. The current statement uses 1, … supplied’

Problem:

You are trying to run a simple SQL query with placeholders on a SQLite3 database, e.g.:

name = "mytable"
conn.execute("SELECT sql FROM sqlite_master WHERE name=?;", name)

But you get an exception like this:

---------------------------------------------------------------------------
ProgrammingError                          Traceback (most recent call last)
<ipython-input-55-e385cf40fd72> in <module>
      1 name = "mytable"
----> 2 conn.execute("SELECT sql FROM sqlite_master WHERE type='table' AND name=?;", name)

ProgrammingError: Incorrect number of bindings supplied. The current statement uses 1, and there are 7 supplied.

Solution:

You need to use a list as the second argument to conn.execute(...)!

Since you only gave the function a string, the string is being interpreted as list of characters.

In our example from above, you just need to wrap name in square brackets to read [name]:

name = "mytable"
conn.execute("SELECT sql FROM sqlite_master WHERE name=?;", [name])
Posted by Uli Köhler in Python, SQLite

How to list SQLite3 database tables on command line

Also see How to list tables in SQLite3 database in Python

Use this command to list all the tables in a SQLite3 database using the sqlite3 command line tool:

sqlite3 [database file] "SELECT name FROM sqlite_master WHERE type='table' AND name != 'sqlite_sequence';"

Example:

$ sqlite3 /usr/share/command-not-found/commands.db "SELECT name FROM sqlite_master WHERE type='table' AND name != 'sqlite_sequence';"
commands
packages

 

Posted by Uli Köhler in Databases, SQLite

How to list tables in SQLite3 database in Python

Also see How to list SQLite3 database tables on command line

You can use this snippet to list all the SQL tables in your SQLite 3.x database in Python:

def tables_in_sqlite_db(conn):
    cursor = conn.execute("SELECT name FROM sqlite_master WHERE type='table';")
    tables = [
        v[0] for v in cursor.fetchall()
        if v[0] != "sqlite_sequence"
    ]
    cursor.close()
    return tables

Usage example:

#!/usr/bin/env python3
import sqlite3
# Open database
conn = sqlite3.connect('/usr/share/command-not-found/commands.db')
# List tables
tables = tables_in_sqlite_db(conn)

# Your code goes here!
# Example:
print(tables) # prints ['commands', 'packages']

 

Posted by Uli Köhler in Databases, Python, SQLite

How to fix ‘apt: command not found’ on Fedora

If you want to install a package on a Fedora Linux, you might have tried a command like

sudo apt install [package name]

However, the Fedora distribution does not use the apt package manager. Fedora uses yum instead.

Use

sudo yum install [package name]

to install a package, for example

sudo yum install sqlite

Note: yum update does not do the same as apt update (i.e. update the list of available packages) but is the equivalent to apt upgrade or apt dist-upgrade i.e. update packages on the system!

Posted by Uli Köhler in Linux

How to dump SQL from SQLite3 database file

If you have a SQLite3 database file (usually the filename extension is .db) and you want to dump it as SQL code, you can use

sqlite3 [database file] .dump

For example:

$ sqlite3 /usr/share/command-not-found/commands.db .dump
[...]
DELETE FROM sqlite_sequence;
INSERT INTO sqlite_sequence VALUES('packages',14669);
INSERT INTO sqlite_sequence VALUES('commands',47706);
CREATE INDEX idx_commands_command ON commands (command);
CREATE INDEX idx_packages_name ON packages (name);
COMMIT;

Missing the sqlite3 executable?

In case you don’t have the sqlite3 executable, you can install it using

sudo apt -y install sqlite3

on Ubuntu/Debian-based system

Posted by Uli Köhler in Allgemein, SQLite

How to install & use nano in a running Docker container

If you want to interactively edit a file in a docker container, you might want to install an editor like GNU nano (for example to debug your config files) in your Docker container that allows you direct access to the container’s file system.

docker exec -it [container name or ID] bash -c 'apt-get -y update && apt -y install nano'

This will work for most debian/Ubuntu-based containers ; for other containers you might need to use a package manager other than apt

Now you can use docker exec -it to interactively edit a config file, e.g.:

docker exec -it [container name or ID] nano /etc/host.conf

 

Posted by Uli Köhler in Container, Docker

What’s the difference between ‘docker exec’ and ‘docker run’?

docker exec runs a program in a container that’s already running. For example, you can use it to create a backup in a container that currently runs your database server.

docker run starts a new container and runs a command in it. For example, your can use it to run a specific script in a container.

Posted by Uli Köhler in Container, Docker

How to automatically cleanup (prune) docker images daily

docker image prune provides an easy way to remove “unused” docker images from a system and hence fixes or significantly delays docker eating up all your disk space on e.g. automated disk space.

provides an easy wayI created a systemd-timer based daily prune routine using TechOverflow’s Simple systemd timer generator.

Quick install using

wget -qO- https://techoverflow.net/scripts/install-cleanup-docker.sh | sudo bash

This is the script which automatically creates & installs both systemd config files.

#!/bin/sh
# This script installs automated docker cleanup via "docker image prune"
# onto systemd-based systems.
# It requires that docker is installed properly

cat >/etc/systemd/system/PruneDocker.service <<EOF
[Unit]
Description=PruneDocker

[Service]
Type=oneshot
ExecStart=/usr/bin/docker image prune -f
WorkingDirectory=/tmp
EOF

cat >/etc/systemd/system/PruneDocker.timer <<EOF
[Unit]
Description=PruneDocker

[Timer]
OnCalendar=daily
Persistent=true

[Install]
WantedBy=timers.target
EOF

# Enable and start service
systemctl enable PruneDocker.timer && sudo systemctl start PruneDocker.timer

To view logs, use

journalctl -xfu PruneDocker.service

To view the status, use

sudo systemctl status PruneDocker.timer

To immediately cleanup your docker images, use

sudo systemctl start PruneDocker.service
Posted by Uli Köhler in Container, Docker

How to remove all unused docker images

Use

docker image prune -f

to remove all unused/old (“dangling”) docker images. -f will skip the confirmation prompt.

This command will remove all images that a) don’t have a tag themselves and b) no image with a tag depends on the image.

Example output:

Deleted Images:
deleted: sha256:b6e0e69c6ba1e811063921d18b52627a9fb905e0bdfd80e226e86958831df636
[...]
deleted: sha256:6a40d04ddf00ad0f1806df7b2d4a2d44a6d8031cab5c369a4bf3d1694d5c48b4

Total reclaimed space: 985.3MB

 

Posted by Uli Köhler in Container, Docker

Lumen to Candela online calculator & Python code

lm

°



Formula

\Omega_{sr} = 2\cdot\pi\cdot(1-\cos(\frac{\theta}{2}))
I_{v} = \frac{\Phi_v}{\Omega_{sr}}

where:

  • \theta is the apex angle in radians
  • \Omega_{sr} is the solid angle in Steradians
  • \Phi_v is the luminous flux in lux (lx).
  • I_{v} is the luminous intensity in candela (cd).

Python code

You can use the UliEngineering library like this:

from UliEngineering.Physics.Light import lumen_to_candela_by_apex_angle
from UliEngineering.EngineerIO import auto_format, auto_print

# These are equivalent:
intensity = lumen_to_candela_by_apex_angle("25 lm", "120°") # intensity = 7.9577 (cd)
intensity = lumen_to_candela_by_apex_angle(25.0, 120.0) # intensity = 7.9577 (cd)

# ... or get out a human-readable value:
intensity_str = auto_format(lumen_to_candela_by_apex_angle, "25 lm", "120°") # "7.96 cd"
# ... or print directly
auto_print(lumen_to_candela_by_apex_angle, "25 lm", "120°") # prints "7.96 cd"

In case you can’t use UliEngineering, use this Python function:

import math

def lumen_to_candela_by_apex_angle(flux, angle):
    """
    Compute the luminous intensity from the luminous flux,
    assuming that the flux of <flux> is distributed equally around
    a cone with apex angle <angle>.

    Keyword parameters
    ------------------
    flux : value, engineer string or NumPy array
        The luminous flux in Lux.
    angle : value, engineer string or NumPy array
        The apex angle of the emission cone, in degrees
        For many LEDs, this is 

    >>> lumen_to_candela_by_apex_angle(25., 120.)
    7.957747154594769
    """
    solid_angle = 2*math.pi*(1.-math.cos((angle*math.pi/180.)/2.0))
    return flux / solid_angle

# Usage example
print(lumen_to_candela_by_apex_angle(25., 120.)) # Prints 7.957747154594769 (cd)

 

Posted by Uli Köhler in Calculators, Physics

How to set Access-Control-Allow-Origin * in nginx

To allow access from all origins, use

add_header 'Access-Control-Allow-Origin' '*' always;

You might want to add

add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';

as well.

Posted by Uli Köhler in nginx

How to emulate the Enter key in Puppeteer

To emulate a keypress to the Enter key in Puppeteer, use

await page.keyboard.press("Enter");

The E needs to be uppercase for this to work!

Posted by Uli Köhler in Javascript, Puppeteer

How to emulate keyboard input in Puppeteer

To emulate the user typing something on the keyboard, use

await page.keyboard.type("the text");

This will type the text extremely fast with virtually no delay between the characters.

In order to simulate the finite typing speed of real users, use

await page.keyboard.type("the text", {delay: 100});

instead. The delay betwen characters in this example is 100 Milliseconds, i.e. the emulated user types 10 characters per second.

Posted by Uli Köhler in Javascript, NodeJS, Puppeteer

How to emulate TAB key press in Puppeteer

In order to emulate a tab key press in Puppeteer, use

await page.keyboard.press("Tab");

Full example:

// Minimal puppeteer example
const puppeteer = require('puppeteer');
(async () => {
  const browser = await puppeteer.launch({defaultViewport: {width: 1920, height: 1080}});
  const page = await browser.newPage();
  await page.goto('https://techoverflow.net', {waitUntil: 'domcontentloaded'});
  // Press tab 10 times (effectively scrolls down on techoverflow.net)
  for (let i = 0; i < 10; i++) {
      await page.keyboard.press("Tab");
  }
  // Screenshot to verify result
  await page.screenshot({path: 'screenshot.png'});
  // Cleanup
  await browser.close();
})();

 

Posted by Uli Köhler in Javascript, Puppeteer