Python

How to automatically remove duplicate docker networks

docker-compose based setups with locally mounted volumes have very few common failure modes in practice. The most important ones are system upgrades to docker stopping all the services and duplicate networks with the same name preventing the startup of a service. Sometimes, docker-compose does not delete the old network properly, possibly due to unclean or unfinished shutdown procedures.

This will result in log messages such as

May 22 21:52:15 myserver docker-compose[2384642]: Removing network etherpad-mydomain_default
May 22 21:52:15 myserver docker-compose[2384642]: network etherpad-mydomain_default is ambiguous (2 matches found based on name)
May 22 21:52:16 myserver systemd[1]: etherpad-mydomain.service: Control process exited, code=exited, status=1/FAILURE

This simple script will find all duplicate network names and simply delete one of them.

#!/usr/bin/env python3
import subprocess
import json

already_seen_networks = set()

output = subprocess.check_output(["docker", "network", "ls", "--format", "{{json .}}"])
for line in output.decode("utf-8").split("\n"):
    line = line.strip()
    if not line: continue
    obj = json.loads(line.strip())
    id = obj["ID"]
    name = obj["Name"]
    if name in already_seen_networks:
        print(f"Detected duplicate network {name}. Removing duplicate network {id}...")
        subprocess.check_output(["docker", "network", "rm", id])

    already_seen_networks.add(name)

Just call this script without any arguments

python docker-remove-duplicate-networks.py

 

Posted by Uli Köhler in Docker, Python

How to compute the weight of a titanium or stainless steel rod using UliEngineering in Python

In the following example, we’ll compute the weight of a pure (i.e. grade 2) titanium rod of dimensions Ø10mm x 100mm and compare it to a 1.4404 steel rod of the same size.

In order to do that, we’ll use UliEngineering.Physics.Density containing pre-defined density values and UliEngineering.Math.Geometry.Cylinder to compute the volume of the rod.

In order to run this script, first install UliEngineering.

from UliEngineering.Math.Geometry.Cylinder import *
from UliEngineering.Physics.Density import *

weight_14404 = cylinder_weight_by_diameter(diameter=0.010, length=0.1, density=Densities['1.4404'])
weight_titanium = cylinder_weight_by_diameter(diameter=0.010, length=0.1, density=Densities['Titanium'])

print(f"Weight of 1.4404 rod: {weight_14404:.3n} kg")
print(f"Weight of Titanium rod: {weight_titanium:.3n} kg")

This prints:

Weight of 1.4404 rod: 0.0628 kg
Weight of Titanium rod: 0.0354 kg

 

Posted by Uli Köhler in Python

How to read KiCAD XML BOM using Python

This script reads an XML BOM (also called python-bom in kicad-cli) using beautifulsoup4 and – as an usage example, generates a RefDes to footprint name dictionary:

from bs4 import BeautifulSoup

def extract_footprints_by_ref_from_xml(filename):
    footprints_by_ref = {} # e.g. "C3" => "Capacitor_SMD:C_0603_1608Metric"
    
    with open(filename, "r") as infile:
        soup = BeautifulSoup(infile, features="xml")
        
    for component in soup.export.components.findChildren("comp"):
        refdes = component.attrs["ref"]
        footprints_by_ref[refdes] = component.footprint.text
    
    return footprints_by_ref

 

Posted by Uli Köhler in KiCAD, Python

How to iterate footprint pads using KiCAD pcbnew plugin Python API

Iterate the pads of a given pcbnew.FOOTPRINT object using

for pad in list(footprint.Pads()):
    # Example of what to do with [pad]
    print(f"{footprint.GetReference()} {pad.GetNetname()}")

Example of how to print all pads for every footprint on the board:

for footprint in list(pcbnew.GetBoard().GetFootprints()):
    for pad in list(footprint.Pads()):
        # Example of what to do with [pad]
        print(f"{footprint.GetReference()} {pad.GetNetname()}")

This will print, for example:

D1 Net-(D1-K)

 

Posted by Uli Köhler in KiCAD, Python

How to get all footprints on board using KiCAD plugin Python API

Use the following for loop to iterate all footprints in the current board:

for footprint in list(pcbnew.GetBoard().GetFootprints()):
    # Example of what to do with [footprint]
    print(footprint.GetReference())

 

Posted by Uli Köhler in KiCAD, Python

How to get all selected zones using KiCAD pcbnew plugin Python API

In our previous post, we showed how to get all selected objects using the KiCAD python API using

pcbnew.GetCurrentSelection()

You can simply filter these entries to obtain just a list of selected zones using either a for loop with inline filtering:

for selected_object in pcbnew.GetCurrentSelection():
    if type(selected_object).__name__ == 'ZONE':
        print(selected_object.GetReference())

or using a list comprehension:

selected_footprints: list[pcbnew.FOOTPRINT] = [
    zone for zone in pcbnew.GetCurrentSelection() if type(zone).__name__ == 'ZONE'
]

Complete plugin example:

#!/usr/bin/env python
import pcbnew
import os

class SimplePlugin(pcbnew.ActionPlugin):
    def defaults(self):
        self.name = "Plugin Name as shown in Pcbnew: Tools->External Plugins"
        self.category = "A descriptive category name"
        self.description = "A description of the plugin and what it does"
        self.show_toolbar_button = False # Optional, defaults to False
        self.icon_file_name = os.path.join(os.path.dirname(__file__), 'simple_plugin.png') # Optional, defaults to ""

    def Run(self):
        board: pcbnew.BOARD = pcbnew.GetBoard()
        footprints: list[pcbnew.FOOTPRINT] = board.GetFootprints()
        
        # TODO Do something useful with [board]
        for selected_object in pcbnew.GetCurrentSelection():
            print(selected_object)

SimplePlugin().register() # Instantiate and register to Pcbnew

Example output (excerpt):

D39
D32
D23
D37
D18
D34
D11
D15

 

Posted by Uli Köhler in KiCAD, Python

KiCAD minimal pcbnew plugin example

The following python script is more or less the minimal example of a KiCAD pcbnew plugin from our previous post How to show wx dialog message in KiCAD pcbnew plugin.

#!/usr/bin/env python
import pcbnew
import wx

class DialogExamplePlugin(pcbnew.ActionPlugin):
    def defaults(self):
        self.name = "Show dialog example"
        self.category = "A descriptive category name"
        self.description = "A description of the plugin and what it does"
        self.show_toolbar_button = False # Optional, defaults to False

    def Run(self):
        dlg = wx.MessageDialog(None, "Please select one or multiple footprints!\n...or use Ctrl+A to select everything.", "No footprints selected", wx.OK | wx.ICON_ERROR)
        dlg.ShowModal()
        dlg.Destroy()

DialogExamplePlugin().register() # Instantiate and register to Pcbnew

You can place this plugin, for example, in

~/.local/share/kicad/7.0/scripting/plugins/DialogExamplePlugin.py

Don’t forget to refresh the plugins from the pcbnew menu.

Posted by Uli Köhler in KiCAD, Python

How to show wx dialog message in KiCAD pcbnew plugin

When using KiCAD’s Python API for pcbnew, you can show a dialog by using the following snippet

# Show info dialog
dlg = wx.MessageDialog(None, "Please select one or multiple footprints!\n...or use Ctrl+A to select everything.", "No footprints selected", wx.OK | wx.ICON_ERROR)
dlg.ShowModal()
dlg.Destroy()

Note that you need to

import wx

at the top of your plugin.

This code will show the following dialog:

 

Complete plugin example:

#!/usr/bin/env python
import pcbnew
import wx

class DialogExamplePlugin(pcbnew.ActionPlugin):
    def defaults(self):
        self.name = "Show dialog example"
        self.category = "A descriptive category name"
        self.description = "A description of the plugin and what it does"
        self.show_toolbar_button = False # Optional, defaults to False

    def Run(self):
        dlg = wx.MessageDialog(None, "Please select one or multiple footprints!\n...or use Ctrl+A to select everything.", "No footprints selected", wx.OK | wx.ICON_ERROR)
        dlg.ShowModal()
        dlg.Destroy()

DialogExamplePlugin().register() # Instantiate and register to Pcbnew

You can place this plugin, for example, in

~/.local/share/kicad/7.0/scripting/plugins/DialogExamplePlugin.py

Don’t forget to refresh the plugins from the pcbnew menu.

Posted by Uli Köhler in KiCAD, Python

How to read KiCAD pick&place position file using pandas in Python

If you’ve exported a KiCAD pick & place position file using the GUI or the command line:

kicad-cli pcb export pos MyPCB.kicad_pcb --units mm -o MyPCB.pos

you can read it from within your Python script using pandas.read_table(...) like this:

import pandas as pd

pos = pd.read_table('KKS-Microcontroller-Board-R2.2.pos', delim_whitespace=True, names=["Ref", "Val", "Package", "PosX", "PosY", "Rot","Side"], comment="#")

Optionally, you can also index pos by the Ref column (which contains values such as C12D1 or U5):

pos.set_index("Ref", inplace=True)

You can also pack all that into a function:

def read_kicad_pos_file(filename):
    pos = pd.read_table(filename, delim_whitespace=True, names=["Ref", "Val", "Package", "PosX", "PosY", "Rot","Side"], comment="#")
    pos.set_index("Ref", inplace=True)
    return pos

If you’ve used .set_index(), you can access a component such as C13 using

pos.loc["C13"]

Example output:

Val                100nF_25V
Package    C_0603_1608Metric
PosX                187.1472
PosY               -101.8243
Rot                    180.0
Side                     top
Name: C1, dtype: object

 

Posted by Uli Köhler in KiCAD, pandas, Python

How to fix Python “match” statement: SyntaxError: invalid syntax

Problem:

Your Python script crashes with SyntaxError: invalid syntax at a match statement, e.g.:

  File "main.py", line 234
    match param:
          ^
SyntaxError: invalid syntax

Solution:

Match statement exist since Python 3.10 – you are using a Python 3.9 or earlier version. You either need to upgrade your Python to 3.10 or later, or rewrite the code and remove the match statement!

Posted by Uli Köhler in Python

How to compute Buck/Boost/LDO output voltage by feedback resistors using Python

You can use the UliEngineering library in order to compute the output voltage of your voltage converter.

First, you need to know Vfb, the feedback voltage of your converter IC, which you need to read from the datasheet. For this example, we’ll use the AP3012‘s feedback voltage of 1.25V.

from UliEngineering.Electronics.VoltageDivider import feedback_actual_voltage
from UliEngineering.EngineerIO import auto_format


# Compute voltage
output_voltage = feedback_actual_voltage("220k", "12.1k", vfb="1.25V")
# output_voltage == 23.97727272727273

# ... or format and print it
auto_format(feedback_actual_voltage, "220k", "12.1k", vfb="1.25V") #  Prints "24.0V"

 

Posted by Uli Köhler in Electronics, Python

How to install pip (Python) on pfSense

Once you have installed python on your pfSense, you might notice that it is missing pip:

[2.6.0-RELEASE][ro[email protected]]/root: pip
pip: Command not found.

and also python3.8 -m pip doesn’t work:

[2.6.0-RELEASE][[email protected]]/root: python3.8 -m pip
/usr/local/bin/python3.8: No module named pip

Installing it is rather easy, however:

python3.8 -m ensurepip

After that, you can run pip using

python3 -m pip

 

Posted by Uli Köhler in Networking, Python

How to install python3 on pfSense

First, login to your pfSense as root using ssh.

Then use

pkg search python

to show which python versions are available. Output on pfSense 2.6.0:

frr7-pythontools-7.5.1_3       Provide configuration reload functionality for FRR
py38-gitpython-3.1.24          Python Git Library
python38-3.8.12_1              Interpreted object-oriented programming language

Now run e.g.

pkg install python38-3.8.12_1

On my pfSense, python3.8 was already installed.

Remember that in order to run python, you need to explicitly run python3.8, just running python or python3 won’t work!

Posted by Uli Köhler in Networking, Python

How to fix PyVISA ValueError: Please install PyUSB to use this resource type

Problem:

When trying to run a Python script using PyVISA with a USB instrument, you see the following error message:

ValueError: Please install PyUSB to use this resource type.
No module named 'usb'

Solution:

On Linux distributions such as Ubuntu, you can fix this using

sudo apt-get -y install python3-usb

This will install the Python3 usb package and the libusb system packages.

Posted by Uli Köhler in Python

How to format axis as dB (decibel) using matplotlib

In our previous post Matplotlib custom SI-prefix unit tick formatter we showed how to format a matplotlib Y axis with a custom unit with SI prefixes.

Similarly, we can use UliEngineering.Math.Decibel in order to format plots (most notably plots with a logarithmic Y axis) as decibels:

from UliEngineering.Math.Decibel import *
import matplotlib.ticker as mtick

def decibel_formatter(v0=1.0, unit='dB'):
    def format_value(value, pos=None):
        dB = value_to_dB(value, v0=v0)
        return f'{dB:.0f} {unit}'
    return format_value

# Usage example:
plt.gca().set_yscale("log") # Optional

plt.gca().yaxis.set_major_formatter(mtick.FuncFormatter(decibel_formatter()))

 

Posted by Uli Köhler in Python

How to fix erpc ValueError: The generated shim code version … is different to the rest of eRPC code

Problem:

When trying to import your erpc project generated Python code using e.g.

from erpc_myproject import *

You see the following error message:

Traceback (most recent call last)
Cell In [1], line 1
----> 1 from erpc_myproject import *

File ./erpc_myproject/__init__.py:13
     11     version = "unknown"
     12 if version != "1.9.1":
---> 13     raise ValueError("The generated shim code version (1.9.1) is different to the rest of eRPC code (%s). \
     14 Install newer version by running \"python setup.py install\" in folder erpc/erpc_python/." % repr(version))
     16 from . import common
     17 from . import client

ValueError: The generated shim code version (1.9.1) is different to the rest of eRPC code ('unknown'). Install newer version by running "python setup.py install" in folder erpc/erpc_python/.

Solution:

Either you have not installed the erpc Python library (if the error message lists ... different to the rest of eRPC code ('unknown')) or you have installed the wrong version (e.g. ... (1.9.1) is different to the rest of eRPC code ('1.10.0')).

If you have not installed erpc at all, simply use

pip install erpc

and retry running your script.

If you have installed the wrong version, you have two options:

Option 1 (preferred): Re-generate the code

Just use the original command (some erpcgen call) you’ve used to re-generate the code using the currently installed version.

Option 2: Install the correct version

For this, you need to determine what the correct version is. Let’s consider the following error message:

ValueError: The generated shim code version (1.9.1) is different to the rest of eRPC code ('1.10.0'). Install newer version by running "python setup.py install" in folder erpc/erpc_python/.

From this message, we can read that the shim code version is 1.9.1 whereas you have 1.10.0 installed. Therefore, in order to make it work, we need to install erpc version 1.9.1.

Install it using

pip install -U erpc==1.9.1

and then retry your command. If you are using jupyter notebooks or similar, you need to restart your kernel to load the new library!

Posted by Uli Köhler in Embedded, Python

How to iterate all IP addresses in network using Python

The following code will iterate over all IP addresses in the given network, i.e. 192.168.1.0 ... 192.168.1.254:

import ipaddress

network = ipaddress.ip_network('192.168.1.0/24')

for ip in network:
    print(ip)

The following variant will iterate over all IP addresses in this network except the broadcast IP address 192.168.1.255 and the network IP address 192.168.1.0:

import ipaddress

network = ipaddress.ip_network('192.168.1.0/24')

for ip in network:
    # Ignore e.g. 192.168.1.0 and 192.168.1.255
    if ip == network.broadcast_address or ip == network.network_address:
        continue
    print(ip)

 

Posted by Uli Köhler in Python

How to fix pip fatal error: portaudio.h: No such file or directory

Problem:

While trying to install a package using pip, you see the following error message:

x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/usr/local/include -I/usr/include -I/usr/include/python3.10 -c src/pyaudio/device_api.c -o build/temp.linux-x86_64-3.10/src/pyaudio/device_api.o
src/pyaudio/device_api.c:9:10: fatal error: portaudio.h: No such file or directory
    9 | #include "portaudio.h"
      |          ^~~~~~~~~~~~~
compilation terminated.
error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
[end of output]

Solution:

On Ubuntu, you need to install the portaudio19-dev package:

sudo apt -y install portaudio19-dev

 

Posted by Uli Köhler in Python

How to compute voltage divider output voltage using Python

You can use the UliEngineering library to compute the output voltage of a voltage divider by its top resistor, bottom resistor and its input voltage:

from UliEngineering.Electronics.VoltageDivider import *

# Voltage divider with 1MΩ and 150kΩ resistors & 24V input voltage
print(voltage_divider_voltage("1MΩ", "150kΩ", "24V")) # Print 3.1304347826086953

You can also directly format this voltage using auto_format():

from UliEngineering.EngineerIO import auto_format

# Print formatted as volts
print(auto_format(voltage_divider_voltage, "1MΩ", "150kΩ", "24V")) # Prints "3.13 V"
Posted by Uli Köhler in Electronics, Python