How to fix ‘apt: command not found’ on Fedora

If you want to install a package on a Fedora Linux, you might have tried a command like

sudo apt install [package name]

However, the Fedora distribution does not use the apt package manager. Fedora uses yum instead.

Use

sudo yum install [package name]

to install a package, for example

sudo yum install sqlite

Note: yum update does not do the same as apt update (i.e. update the list of available packages) but is the equivalent to apt upgrade or apt dist-upgrade i.e. update packages on the system!

Posted by Uli Köhler in Linux

How to dump SQL from SQLite3 database file

If you have a SQLite3 database file (usually the filename extension is .db) and you want to dump it as SQL code, you can use

sqlite3 [database file] .dump

For example:

$ sqlite3 /usr/share/command-not-found/commands.db .dump
[...]
DELETE FROM sqlite_sequence;
INSERT INTO sqlite_sequence VALUES('packages',14669);
INSERT INTO sqlite_sequence VALUES('commands',47706);
CREATE INDEX idx_commands_command ON commands (command);
CREATE INDEX idx_packages_name ON packages (name);
COMMIT;

Missing the sqlite3 executable?

In case you don’t have the sqlite3 executable, you can install it using

sudo apt -y install sqlite3

on Ubuntu/Debian-based system

Posted by Uli Köhler in Allgemein, SQLite

How to install & use nano in a running Docker container

If you want to interactively edit a file in a docker container, you might want to install an editor like GNU nano (for example to debug your config files) in your Docker container that allows you direct access to the container’s file system.

docker exec -it [container name or ID] bash -c 'apt-get -y update && apt -y install nano'

This will work for most debian/Ubuntu-based containers ; for other containers you might need to use a package manager other than apt

Now you can use docker exec -it to interactively edit a config file, e.g.:

docker exec -it [container name or ID] nano /etc/host.conf

 

Posted by Uli Köhler in Container, Docker

What’s the difference between ‘docker exec’ and ‘docker run’?

docker exec runs a program in a container that’s already running. For example, you can use it to create a backup in a container that currently runs your database server.

docker run starts a new container and runs a command in it. For example, your can use it to run a specific script in a container.

Posted by Uli Köhler in Container, Docker

How to automatically cleanup (prune) docker images daily

Note: This will only remove docker images without a tag, but not all images not associated to a running or stopped container. See our post on How to automatically cleanup (prune) docker images daily in case this is not the desired behaviour.

docker image prune provides an easy way to remove “unused” (i.e. untagged) docker images from a system and hence fixes or significantly delays docker eating up all your disk space on.

I created a systemd-timer based daily prune routine using TechOverflow’s Simple systemd timer generator.

Quick install using

wget -qO- https://techoverflow.net/scripts/install-cleanup-docker.sh | sudo bash

This is the script which automatically creates & installs both systemd config files.

#!/bin/sh
# This script installs automated docker cleanup via "docker image prune"
# onto systemd-based systems.
# It requires that docker is installed properly

cat >/etc/systemd/system/PruneDocker.service <<EOF
[Unit]
Description=PruneDocker

[Service]
Type=oneshot
ExecStart=/usr/bin/docker image prune -f
WorkingDirectory=/tmp
EOF

cat >/etc/systemd/system/PruneDocker.timer <<EOF
[Unit]
Description=PruneDocker

[Timer]
OnCalendar=daily
Persistent=true

[Install]
WantedBy=timers.target
EOF

# Enable and start service
systemctl enable PruneDocker.timer && sudo systemctl start PruneDocker.timer

To view logs, use

journalctl -xfu PruneDocker.service

To view the status, use

sudo systemctl status PruneDocker.timer

To immediately cleanup your docker images, use

sudo systemctl start PruneDocker.service
Posted by Uli Köhler in Container, Docker

How to remove all unused docker images

Use

docker image prune -f

to remove all unused/old (“dangling”) docker images. -f will skip the confirmation prompt.

This command will remove all images that a) don’t have a tag themselves and b) no image with a tag depends on the image.

Example output:

Deleted Images:
deleted: sha256:b6e0e69c6ba1e811063921d18b52627a9fb905e0bdfd80e226e86958831df636
[...]
deleted: sha256:6a40d04ddf00ad0f1806df7b2d4a2d44a6d8031cab5c369a4bf3d1694d5c48b4

Total reclaimed space: 985.3MB

 

Posted by Uli Köhler in Container, Docker

Lumen to Candela online calculator & Python code

TechOverflow calculators:
You can enter values with SI suffixes like 12.2m (equivalent to 0.012) or 14k (14000) or 32u (0.000032).
The results are calculated while you type and shown directly below the calculator, so there is no need to press return or click on a Calculate button. Just make sure that all inputs are green by entering valid values.

lm

°



Formula

\Omega_{sr} = 2\cdot\pi\cdot(1-\cos(\frac{\theta}{2}))
I_{v} = \frac{\Phi_v}{\Omega_{sr}}

where:

  • \theta is the apex angle in radians
  • \Omega_{sr} is the solid angle in Steradians
  • \Phi_v is the luminous flux in lux (lx).
  • I_{v} is the luminous intensity in candela (cd).

Python code

You can use the UliEngineering library like this:

from UliEngineering.Physics.Light import lumen_to_candela_by_apex_angle
from UliEngineering.EngineerIO import auto_format, auto_print

# These are equivalent:
intensity = lumen_to_candela_by_apex_angle("25 lm", "120°") # intensity = 7.9577 (cd)
intensity = lumen_to_candela_by_apex_angle(25.0, 120.0) # intensity = 7.9577 (cd)

# ... or get out a human-readable value:
intensity_str = auto_format(lumen_to_candela_by_apex_angle, "25 lm", "120°") # "7.96 cd"
# ... or print directly
auto_print(lumen_to_candela_by_apex_angle, "25 lm", "120°") # prints "7.96 cd"

In case you can’t use UliEngineering, use this Python function:

import math

def lumen_to_candela_by_apex_angle(flux, angle):
    """
    Compute the luminous intensity from the luminous flux,
    assuming that the flux of <flux> is distributed equally around
    a cone with apex angle <angle>.

    Keyword parameters
    ------------------
    flux : value, engineer string or NumPy array
        The luminous flux in Lux.
    angle : value, engineer string or NumPy array
        The apex angle of the emission cone, in degrees
        For many LEDs, this is 

    >>> lumen_to_candela_by_apex_angle(25., 120.)
    7.957747154594769
    """
    solid_angle = 2*math.pi*(1.-math.cos((angle*math.pi/180.)/2.0))
    return flux / solid_angle

# Usage example
print(lumen_to_candela_by_apex_angle(25., 120.)) # Prints 7.957747154594769 (cd)

 

Posted by Uli Köhler in Calculators, Physics

How to set Access-Control-Allow-Origin * in nginx

To allow access from all origins, use

add_header 'Access-Control-Allow-Origin' '*' always;

You might want to add

add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';

as well.

Posted by Uli Köhler in nginx

How to emulate the Enter key in Puppeteer

To emulate a keypress to the Enter key in Puppeteer, use

await page.keyboard.press("Enter");

The E needs to be uppercase for this to work!

Posted by Uli Köhler in Javascript, Puppeteer

How to emulate keyboard input in Puppeteer

To emulate the user typing something on the keyboard, use

await page.keyboard.type("the text");

This will type the text extremely fast with virtually no delay between the characters.

In order to simulate the finite typing speed of real users, use

await page.keyboard.type("the text", {delay: 100});

instead. The delay betwen characters in this example is 100 Milliseconds, i.e. the emulated user types 10 characters per second.

Posted by Uli Köhler in Javascript, NodeJS, Puppeteer

How to emulate TAB key press in Puppeteer

In order to emulate a tab key press in Puppeteer, use

await page.keyboard.press("Tab");

Full example:

// Minimal puppeteer example
const puppeteer = require('puppeteer');
(async () => {
  const browser = await puppeteer.launch({defaultViewport: {width: 1920, height: 1080}});
  const page = await browser.newPage();
  await page.goto('https://techoverflow.net', {waitUntil: 'domcontentloaded'});
  // Press tab 10 times (effectively scrolls down on techoverflow.net)
  for (let i = 0; i < 10; i++) {
      await page.keyboard.press("Tab");
  }
  // Screenshot to verify result
  await page.screenshot({path: 'screenshot.png'});
  // Cleanup
  await browser.close();
})();

 

Posted by Uli Köhler in Javascript, Puppeteer

How to fix Puppeteer ‘Error: Unknown key: “tab”‘

Problem:

You want to emulate a tab key press using

await page.keyboard.press("tab");

but you get an error message like

(node:30594) UnhandledPromiseRejectionWarning: Error: Unknown key: "tab"
    at assert (/home/uli/dev/myproject/node_modules/puppeteer/lib/helper.js:270:11)
    at Keyboard._keyDescriptionForString (/home/uli/dev/myproject/node_modules/puppeteer/lib/Input.js:96:5)
    at Keyboard.down (/home/uli/dev/myproject/node_modules/puppeteer/lib/Input.js:44:30)
    at Keyboard.<anonymous> (/home/uli/dev/myproject/node_modules/puppeteer/lib/helper.js:112:23)
    at Keyboard.press (/home/uli/dev/myproject/node_modules/puppeteer/lib/Input.js:178:16)
    at Keyboard.<anonymous> (/home/uli/dev/myproject/node_modules/puppeteer/lib/helper.js:112:23)
    at /home/uli/dev/myproject/test.js:8:23
    at processTicksAndRejections (internal/process/task_queues.js:85:5)
(node:30594) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)
(node:30594) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Solution:

You need to use Tab, not tab ! The T needs to be uppercase!

Use

await page.keyboard.press("Tab");

Full example:

// Minimal puppeteer example
const puppeteer = require('puppeteer');
(async () => {
  const browser = await puppeteer.launch({defaultViewport: {width: 1920, height: 1080}});
  const page = await browser.newPage();
  await page.goto('https://techoverflow.net', {waitUntil: 'domcontentloaded'});
  // Press tab 10 times (effectively scrolls down on techoverflow.net)
  for (let i = 0; i < 10; i++) {
      await page.keyboard.press("Tab");
  }
  // Screenshot to verify result
  await page.screenshot({path: 'screenshot.png'});
  // Cleanup
  await browser.close();
})();

 

Posted by Uli Köhler in Javascript

How to set <input> value in Puppeteer

Use this snippet to set the value of an HTML <input> element in Puppeteer:

const newInputValue = "test 123";
await page.evaluate(val => document.querySelector('.search-form-input').value = val, newInputValue);amp

Remember to replace '.search-form-input' by whatever CSS selector is suitable to select your <input>. Examples include 'input[name="username"]' or '.username > input'.

Full example:

// Minimal puppeteer example
const puppeteer = require('puppeteer');
(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('https://techoverflow.net', {waitUntil: 'domcontentloaded'});
  // Set input value
  const newInputValue = "test 123";
  await page.evaluate(val => document.querySelector('.search-form-input').value = val, newInputValue);
  // Screenshot to verify result
  await page.screenshot({path: 'screenshot.png'});
  // Cleanup
  await browser.close();
})();

Note that this method will work for any simple <input>, however it might not work for some heavily Javascripted inputs which you can find on some modern websites.

Posted by Uli Köhler in Javascript, Puppeteer

How to save screenshot in Puppeteer as PNG?

You can take a screenshot in Puppeteer using

await page.screenshot({path: 'screenshot.png'});

The path is relative to the current working directory.

Want to have a screenshot in a size different to 800×600? See How to set screenshot size in Puppeteer?

Full example:

// Minimal puppeteer example
const puppeteer = require('puppeteer');
(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('https://techoverflow.net', {waitUntil: 'domcontentloaded'});
  // Screenshot to verify result
  await page.screenshot({path: 'screenshot.png'});
  // Cleanup
  await browser.close();
})();

 

Posted by Uli Köhler in Javascript, NodeJS, Puppeteer

How to identify large directories for ‘No space left on device’ on Linux

TL;DR

cd / and run

sudo du -sh * --exclude proc --exclude sys --exclude dev

and then repeat for the largest directory shown (by cding to that directory and running the command above)

Long answer

If you get No space left on device errors on Linux, this means that one of your mounted disks has (virtually) no space left to write on.

First, check which device is the one that has no space left:

$ sudo df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            2.4G     0  2.4G   0% /dev
tmpfs           494M   51M  444M  11% /run
/dev/xvda1       46G   17G   11M 100% /
tmpfs           2.5G     0  2.5G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           2.5G     0  2.5G   0% /sys/fs/cgroup
tmpfs           494M     0  494M   0% /run/user/1003

Check the Use % column – you can see that the device mounted on / (your root filesystem, i.e. the one where your system is mounted) is full. In 95% of all cases it’s the root filesystem that’s full.

This post only covers the case where your root filesystem is full. In most other cases, it’s either /dev/shm (in which case rebooting your system typically works) or an external drive (in which case you have to figure out for yourself which directories can be deleted).

Probabilistic method:

This quick check tries to identify the most likely candidates first (based on admin experience). Run this in your shell:

sudo du -sh /var/lib/docker /var/lib/mysql /var/lib/postgresql /home/

This might take some time to complete.

Check if one of those directories is so large that it eats up a significant fraction of your drive space. If not, I recommend going forward with the Simple method:

Simple method:

Copy & paste this into your shell:

function findLargestSubdir { cd "$1" && sudo du -sb * --exclude proc --exclude sys --exclude dev | sort -n ; }

then run

findLargestSubdir /

This will tell you which directory is the largest. It might take a long time to compute the size of all directoriesThis command will only print anything once it’s finished! The numbers are in bytes.

The last element in the list is the largest one!

Now, run the same command inside the largest directory to find the largest sub-directory. For example, if /var happens to be the largest directory, run

findLargestSubdir /var

which will show you the largest directory in /var. Continue checking the largest subdir(s) using findLargestSubdir until you found out what ate up all the space on your disk.

Advanced method: Much quicker but more complex

If it takes more than 2 minutes to compute the size of all directories, I recommend following this interactive procedure as root (sudo):

  1. ls -1 to show all files and directories in /. Example output:
    # ls -1 /
    bin
    boot
    dev
    etc
    home
    initrd.img
    initrd.img.old
    lib
    lib64
    lost+found
    media
    mnt
    opt
    proc
    root
    run
    sbin
    snap
    srv
    sys
    tmp
    usr
    var
    vmlinuz
    vmlinuz.old
  2. Now run du -sh *. This will try to compute the size of each of those files and directories. Typically this command will stall for a long time when trying to compute the size of one directory (if that directory has a huge number of files in it). For example, when you see the output
    # du -sh *
    17M     bin
    157M    boot
    0       dev
    7.7M    etc

    and then nothing happens for more than 30 seconds, look up the next entry after the last entry in the list ( etc in this example) in the output of ls -1 above. In this example, this would be /home. Since du -sh took so long computing the size of /home it’s very likely (though not guaranteed) that /home is the directory that takes up so much space. Also check the number

  3. In whatever directory you found to be a candidate for being the largest directory, run ls -1 <directory> again and check

Note that this method sometimes has a tendency to identify directories that recursively contain many files as opposed to directories whose total size is large. Therefore, you might need to go back in case you can’t identify any directories that eat up a large fraction of your hard drive space.

Posted by Uli Köhler in Linux

How to make your own ‘get my current IP address’ server using only nginx

Note: In some configurations, this returns e.g. ::ffff:1.2.3.4 for IPv4 requests and I did not resolve this yet

This nginx config will return the user’s IP address to the user. Note that it won’t work behind a reverse proxy, it actually needs to listen on port 80 and/or 443 of the IP address of the domain it is served as.

location = /api/get-my-ip {
    add_header 'Access-Control-Allow-Origin' '*' always;
    add_header 'Access-Control-Allow-Methods' 'GET, OPTIONS';
    add_header 'Content-Type' 'text/plain charset=UTF-8';
    return 200 '$remote_addr';
}

This is a JSON-returning variant:

location = /api/get-my-ip-json {
    add_header 'Access-Control-Allow-Origin' '*' always;
    add_header 'Access-Control-Allow-Methods' 'GET, OPTIONS';
    add_header 'Content-Type' 'text/plain charset=UTF-8';
    return 200 '{"ip": "$remote_addr"}';
}

This configuration is available online, for free, at techoverflow.net: See How to get your current IPv4 address using wget and How to get your current IPv6 address using wget

Posted by Uli Köhler in Networking, nginx

How to sleep in NodeJS using async/await

The easiest way to sleep using async/await in NodeJS is the sleep-promise package:

npm install --save sleep-promise

Using that package you can simply use await sleep(milliseconds) syntax like this:

const sleep = require('sleep-promise');

// In any async function:
await sleep(2000); // Wait 2000 ms

Note that this sleep() variant is fully asynchronous, IO and other asynchronous operations will still be able to continue in the background – sleep() will not block the NodeJS process.

Full example:

const sleep = require('sleep-promise');

(async () => {
    console.log("This prints immediately");
    await sleep(2000);
    console.log("This prints 2 seconds later");
})();

 

Posted by Uli Köhler in Javascript, NodeJS

How to fix nginx rewrite redirect causing certbot verification errors

When using a redirection to a new domain using rewrite in my nginx config, I had issues with Let’s Encrypt not being able to verify the domain ownership when using certbot --nginx.

My redirect statement was

location / {
    rewrite ^/$ https://newdomain.com permanent;
    rewrite ^/(.*)$ https://newdomain.com/$1 permanent;
}

I was able to fix the issue by instead using this redirect statement:

location / {
     return 301 https://newdomain.com$request_uri;
}

After using this, I was able to use certbot --nginx without any issues.

Posted by Uli Köhler in nginx

How to redirect to a new domain using nginx

In order to redirect to a different domain using nginx, use this snippet inside your server {...} block:

location / {
     return 301 https://newdomain.com$request_uri;
}

Full example:

server {
    server_name old.domain;
    listen 80;

    location / {
         return 301 https://techoverflow.net$request_uri;
    }
}
Posted by Uli Köhler in nginx

NodeJS equivalent to Python’s time.sleep() using async/await

Install the sleep-promise package:

npm install --save sleep-promise

then you can use

const sleep = require('sleep-promise');

// In any async function:
await sleep(2000); // Wait 2000 ms

Full example:

const sleep = require('sleep-promise');

(async () => {
    console.log("This prints immediately");
    await sleep(2000);
    console.log("This prints 2 seconds later");
})();

 

Posted by Uli Köhler in Javascript, NodeJS
This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish. Cookie settingsACCEPTPrivacy &amp; Cookies Policy