Linux

Robust Linux Python serial port filtering by name and manufacturer using udev

Important note: This code is deprecated as it has been superseded by the UliSerial library on GitHub. UIiSerial implements the serial port filtering in a platform-independent manner using pyserial built-in features.

 

Typically, when one intends to select a serial port in Python, you use a sequence such as

glob.glob('/dev/ttyACM*')[0]

or just go to straight hard-coding the port such as /dev/ttyACM0.

It should be quite obvious why this isn’t robust:

  • There might be multiple serial ports and the order is unknown
  • You code might need to work on different computers, which lead to different ports being assigned
  • When you re-plug a device, it might get assigned a differnent serial port number (e.g. /dev/ttyACM1)

The following code uses only the linux integrated tool udevadm to query information and built-in libraries. Currently it does not work on anything but Linux.

import glob
import subprocess
import re

def get_serial_ports():
    # Get a list of /dev/ttyACM* and /dev/ttyUSB* devices
    ports = glob.glob('/dev/ttyACM*') + glob.glob('/dev/ttyUSB*')
    return ports

def get_udev_info(port):
    # Run the udevadm command and get the output
    result = subprocess.run(['udevadm', 'info', '-q', 'all', '-r', '-n', port], capture_output=True, text=True)
    return result.stdout

def find_serial_ports(vendor=None, model=None, usb_vendor=None, usb_model=None, usb_model_id=None, vendor_id=None):
    # Mapping of human-readable names to udevadm keys
    attribute_mapping = {
        "vendor": "ID_VENDOR_FROM_DATABASE",
        "model": "ID_MODEL_FROM_DATABASE",
        "usb_vendor": "ID_USB_VENDOR",
        "usb_model": "ID_USB_MODEL_ENC",
        "usb_model_id": "ID_USB_MODEL_ID",
        "vendor_id": "ID_VENDOR_ID",
        "usb_path": "ID_PATH",
        "serial": "ID_SERIAL_SHORT"
    }

    # Filters based on provided arguments
    filters = {}
    for key, value in locals().items():
        if value and key in attribute_mapping:
            filters[attribute_mapping[key]] = value

    # Find and filter ports
    ports = get_serial_ports()
    filtered_ports = []

    for port in ports:
        udev_info = get_udev_info(port)
        match = True

        for key, value in filters.items():
            if not re.search(f"{key}={re.escape(value)}", udev_info):
                match = False
                break

        if match:
            filtered_ports.append(port)

    return filtered_ports

def get_serial_port_info(port):
    # Mapping of udevadm keys to human-readable names
    attribute_mapping = {
        "ID_VENDOR_FROM_DATABASE": "vendor",
        "ID_MODEL_FROM_DATABASE": "model",
        "ID_USB_VENDOR": "usb_vendor",
        "ID_USB_MODEL_ENC": "usb_model",
        "ID_USB_MODEL_ID": "usb_model_id",
        "ID_VENDOR_ID": "vendor_id",
        "ID_SERIAL_SHORT": "serial",
    }

    # Run the udevadm command and get the output
    udev_info = get_udev_info(port)
    port_info = {}

    for line in udev_info.splitlines():
        if line.startswith('E: '):
            key, value = line[3:].split('=', 1)
            if key in attribute_mapping:
                # Decode escape sequences like \x20 to a space.
                # NOTE: Since only \x20 is common, we currently only replace that one
                port_info[attribute_mapping[key]] = value.replace('\\x20', ' ')

    return port_info

def find_serial_port(**kwargs):
    """
    Find a single serial port matching the provided filters.
    """
    ports = find_serial_ports(**kwargs)
    if len(ports) > 1:
        raise ValueError("Multiple matching ports found for filters: " + str(kwargs))
    elif len(ports) == 0:
        raise ValueError("No matching ports found for filters: " + str(kwargs))
    else:
        return ports[0]

 

Example: Find a serial port

# Example usage: Find a serial port
matching_ports = find_serial_ports(vendor="OpenMoko, Inc.")
print(matching_ports) # Prints e.g. ['/dev/ttyACM0']

Example: Print information about a given port

This is typically used so you know what filters to add for find_serial_ports().

# Example usage: Print info about a serial port
serial_port_info = get_serial_port_info('/dev/ttyACM0')
print(serial_port_info)

This prints, for example:

{
    'serial': '01010A23535223934CF29A1EF5000007',
    'vendor_id': '1d50',
    'usb_model': 'Marlin USB Device',
    'usb_model_id': '6029',
    'usb_vendor':
    'marlinfw.org',
    'vendor': 'OpenMoko, Inc.',
    'model': 'Marlin 2.0 (Serial)'
}

 

Posted by Uli Köhler in Electronics, Linux, Python

How to install NodeJS 20.x LTS on Ubuntu in 1 minute

Run these shell commands on your Ubuntu computer to install NodeJS 20.x:

sudo apt-get update
sudo apt-get install -y ca-certificates curl gnupg
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | sudo gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg

NODE_MAJOR=20
echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | sudo tee /etc/apt/sources.list.d/nodesource.list

sudo apt-get update
sudo apt-get install nodejs -y

 

Instead of 20 you can also choose other versions like 18. However, using this method, you can’t install multiple versions of NodeJS in parallel.

Source: Official nodesource documentation

Posted by Uli Köhler in Linux, NodeJS

How I fixed SSH X-forwarding error “X11 connection rejected because of wrong authentication”

Problem:

I connected to my server with X forwarding using

ssh -CX user@server

but when I start a graphical application on the server, I see the following error messages:

user@server $ gparted
X11 connection rejected because of wrong authentication.
Unable to init server: Could not connect: Connection refused

(gpartedbin:70357): Gtk-WARNING **: 00:06:19.612: cannot open display: localhost:11.0

Solution:

I found the solution in this StackOverflow post:

sudo --preserve-env=DISPLAY -s
xauth merge /home/user/.Xauthority

Note that you have to replace /home/user with the appropriate home directory.

Posted by Uli Köhler in Linux

How to fix CIFS/SMB mount error: VFS: \\… Send error in SessSetup = -13

Problem:

While trying to mount a SMB share on Linux, the mount command fails with

mount: /nas: cannot mount //10.1.2.3/myshare read-only.

with dmesg showing the following error messages:

[  689.574128] CIFS: Attempting to mount \\10.1.2.3\myshare
[  689.621472] CIFS: Status code returned 0xc000006d STATUS_LOGON_FAILURE
[  689.621483] CIFS: VFS: \\10.1.2.3 Send error in SessSetup = -13
[  689.621503] CIFS: VFS: cifs_mount failed w/return code = -13

Solution:

Error -13 means that you are missing the cifs-utils package.

Install it using

sudo apt -y install cifs-utils

 

Posted by Uli Köhler in Linux

How to remove all files containing a given string recursively using the Linux shell

This command will remove all text files containg abc123abc recursively:

ag abc123abc --hidden -l | xargs rm -v

I have not tested with filenames containing spaces etc. The files will be deleted permanently, so try it out first using just

ag abc123abc --hidden -l

 

Posted by Uli Köhler in Linux

How to fix unixODBC “Can’t open lib ‘postgresql’: file not found” on Linux

Problem:

When you try to connect to a PostgreSQL database using a ODBC application such as KiCAD (database library connection), you see the following error message:

[unixODBC][Driver Manager]Can't open lib 'postgresql' : file not found

Solution:

First, install the ODBC PostgreSQL driver adapter:

sudo apt -y install odbc-postgresql

Using that driver, you would typically use a driver setting such as

Driver={PostgreSQL Unicode}

 

Posted by Uli Köhler in Databases, KiCAD, Linux

How to fix APT error NO_PUBKEY B53DC80D13EDEF05

Problem:

When you run apt update, you see the following error message:

Err:12 http://packages.cloud.google.com/apt gcsfuse-bionic InRelease
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY B53DC80D13EDEF05

Solution:

Install the Google Cloud key using

curl -fsSL https://packages.cloud.google.com/apt/doc/apt-key.gpg| gpg -o /usr/share/keyrings/kubernetes-archive-keyring.gpg --dearmor

After that, you can try running

sudo apt update

again.

Posted by Uli Köhler in Linux

How to install /etc/services on Ubuntu Docker image

When you use a Ubuntu-based docker image such as

FROM ubuntu:22.04

by default, /etc/services is not installed.

However, you can easily install it by installing the netbase apt package:

# netbase provides /etc/services
RUN apt update && apt install -y netbase && rm -rf /var/lib/apt/lists/*

 

Posted by Uli Köhler in Docker, Linux

How to install uvcdynctrl on Raspberry Pi OS Bullseye

Problem

You are trying to install uvcdynctrl on Raspberry Pi OS bullseye version using

sudo apt -y install uvcdynctrl

the system can’t find the package:

Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
E: Unable to locate package uvcdynctrl

Solution

The uvcdynctrl package is not available in the standard Raspberry Pi OS repository, but only in the bullseye-backports repository.

In order to solve the problem, activate the backports repository:

Add the following lines at the end of /etc/apt/sources.list:

deb http://deb.debian.org/debian bullseye-backports main contrib non-free
deb-src http://deb.debian.org/debian bullseye-backports main contrib non-free

After that, run

sudo apt update

and finally, try to install uvcdynctrl again:

sudo apt -y install uvcdynctrl

 

 

Posted by Uli Köhler in Linux, Raspberry Pi

How to activate backports repository on Raspberry Pi OS (Raspbian)

Add the following lines at the end of /etc/apt/sources.list:

deb http://deb.debian.org/debian bullseye-backports main contrib non-free
deb-src http://deb.debian.org/debian bullseye-backports main contrib non-free

After that, run

sudo apt update

 

Posted by Uli Köhler in Linux, Raspberry Pi

How to install Signal Desktop on Ubuntu 22.04

Use the following command to install Signal Desktop:

sudo snap install signal-desktop

 

Posted by Uli Köhler in Linux

How to split multi-page TIFF using Linux shell and convert to PNG

convert mytiff.tiff -format png -scene 1 out%d.png

This will produce out1.pngout2.png, …

Optionally, you can also rotate the files:

convert mytiff.tiff -rotate 90 -format png -scene 1 out%d.png

 

Posted by Uli Köhler in Linux

How to recursively remove executable (x) bit from files

This simple shell script will remove the executable bit (chmod -x) from files (but not directories) recursively:

find . -type f -exec chmod -x {} \;

 

Posted by Uli Köhler in Linux

How to install MinIO commander (mc) globally on Linux

sudo bash -c 'curl https://dl.min.io/client/mc/release/linux-amd64/mc > /usr/local/bin/mc'
sudo chmod a+x /usr/local/bin/mc

 

Posted by Uli Köhler in Linux, S3

How to connect to Tekceleo WLG-75-R on Linux using picocom

Use the following command to connect to the Tekceleo WLG-75-R piezo motor eval kit using the picocom terminal program:

picocom -b 9600 /dev/ttyACM0 --imap lfcrlf --echo

 

Posted by Uli Köhler in Linux

How to query USB device information using udev

You can use udevadm to query information such as the USB device path for a given USB device such as /dev/usbtmc0:

udevadm info -q all /dev/usbtmc0

Example output:

P: /devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/usb1/1-1/1-1.3/1-1.3:1.0/usbmisc/usbtmc0
N: usbtmc0
L: 0
E: DEVPATH=/devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/usb1/1-1/1-1.3/1-1.3:1.0/usbmisc/usbtmc0
E: DEVNAME=/dev/usbtmc0
E: MAJOR=180
E: MINOR=176
E: SUBSYSTEM=usbmisc

 

Posted by Uli Köhler in Linux

Thorlabs PM100(D) USB-TMC udev rules

SUBSYSTEMS=="usb", ACTION=="add", ATTRS{idVendor}=="1313", ATTRS{idProduct}=="8078", GROUP="dialout", MODE="0666"

How to install:

sudo cp 99-thorlabs-pm100.rules /etc/udev/rules.d/
sudo udevadm control --reload-rules && sudo udevadm trigger

 

Posted by Uli Köhler in Linux

How to set Linux hardware clock to a specific date

hwclock --set --date '2021-01-04 13:04:00'

 

Posted by Uli Köhler in Linux

Systemd service to use a DS3231 RTC on the Raspberry Pi

The following systemd service will automatically. See this guide for more information on the setup and ensure sudo i2cdetect -y 1 detects the RTC with address 0x68.

This is an automatic service installation & enable script based on A simple systemd service autoinstall script . This script will automatically enable the service on boot:

#!/bin/bash
# This script installs and enables/starts a systemd service
# It also installs the service file
export NAME=ConfigureRTC

cat >/etc/systemd/system/${NAME}.service <<EOF
[Unit]
Description=${NAME}

[Service]
ExecStart=/bin/bash -c 'echo ds1307 0x68 > /sys/class/i2c-adapter/i2c-1/new_device && hwclock -s'
Restart=always

[Install]
WantedBy=multi-user.target
EOF

# Enable and start service
systemctl enable --now ${NAME}.service

This is just the systemd service:

[Unit]
Description=ConfigureRTC

[Service]
ExecStart=/bin/bash -c 'echo ds1307 0x68 > /sys/class/i2c-adapter/i2c-1/new_device && hwclock -s'
Restart=always

[Install]
WantedBy=multi-user.target

 

Posted by Uli Köhler in Linux, Raspberry Pi