Python

How to create constant voltage source in PySpice

This creates a voltage source named V1 which has its + pin connected to node n1 and its - pin connected to GND. The voltage source is set to 5V DC.

source = circuit.VoltageSource('V1', 'n1', circuit.gnd, dc_value=5@u_V)

Full example:

import PySpice.Logging.Logging as Logging
logger = Logging.setup_logging()

from PySpice.Probe.Plot import plot
from PySpice.Spice.Netlist import Circuit
from PySpice.Unit import *

circuit = Circuit("MyCircuit")
source = circuit.VoltageSource('V1', 'n1', circuit.gnd, dc_value=5@u_V)

 

Posted by Uli Köhler in Electronics, Python, SPICE

How to replace numpy NaNs by last non-NaN value

Use bottleneck.push:

import bottleneck

y = bottleneck.push(y, axis=0)

If not installed already, you can install bottleneck using

pip install bottleneck

 

Posted by Uli Köhler in Python

How to set matplotlib X axis timestamp format to hours and minutes

import matplotlib.dates as mdates

plt.gca().xaxis.set_major_formatter(mdates.DateFormatter('%H:%M'))

 

Posted by Uli Köhler in Python

How to change matplotlib X axis timestamp format

Sometimes you get weird timestamp format choices in Matplotlib such as 22 19:40 (meaning 19:40 o’clock on the 22nd of whatever month).

This is easy to fix:

import matplotlib.dates as mdates

plt.gca().xaxis.set_major_formatter(mdates.DateFormatter('%H:%M'))

which will result in this image, for example:

Posted by Uli Köhler in Python

LinuxCNC: How to get current position including offset using Python

In our previous example LinuxCNC: How to find current position using Python we showed how to use stat.actual_position to get the current position in machine coordinates using LinuxCNC’s Python API.

#!/usr/bin/env python2.7
import linuxcnc

stat = linuxcnc.stat()
stat.poll()
x,y,z,a,b,c,u,v,w = stat.actual_position

# NOTE: Ignore ABCUVW since not used for my machine
# Subtract G5x offset
xo,yo,zo,ao,bo,co,uo,vo,wo = stat.g5x_offset

x -= xo
y -= yo
z -= zo

# Subtract tool offset
xo,yo,zo,ao,bo,co,uo,vo,wo = stat.tool_offset

x -= xo
y -= yo
z -= zo

# Print offset coordinates
print(x,y,z)

 

Posted by Uli Köhler in LinuxCNC, Python

LinuxCNC: What attributes can you read from a linuxcnc.stat() object?

As shown in our previous example LinuxCNC: How to find current position using Python you can get an

These are the attributes of the object returned by linuxcnc.stat() for LinuxCNC 2.7:

acceleration
active_queue
actual_position
adaptive_feed_enabled
ain
angular_units
aout
axes
axis
axis_mask
block_delete
call_level
command
current_line
current_vel
cycle_time
debug
delay_left
din
distance_to_go
dout
dtg
echo_serial_number
enabled
estop
exec_state
feed_hold_enabled
feed_override_enabled
feedrate
file
flood
g5x_index
g5x_offset
g92_offset
gcodes
homed
id
inpos
input_timeout
interp_state
interpreter_errcode
joint_actual_position
joint_position
kinematics_type
limit
linear_units
lube
lube_level
max_acceleration
max_velocity
mcodes
mist
motion_line
motion_mode
motion_type
optional_stop
paused
pocket_prepped
poll
position
probe_tripped
probe_val
probed_position
probing
program_units
queue
queue_full
queued_mdi_commands
rapidrate
read_line
rotation_xy
settings
spindle_brake
spindle_direction
spindle_enabled
spindle_increasing
spindle_override_enabled
spindle_speed
spindlerate
state
task_mode
task_paused
task_state
tool_in_spindle
tool_offset
tool_table
velocity

You can find those out using

#!/usr/bin/env python2.7
import linuxcnc

stat = linuxcnc.stat()
stat.poll()
for entry in dir(stat):
    print(entry)

 

Posted by Uli Köhler in LinuxCNC, Python

LinuxCNC: How to find current position using Python

This will show the position in machine coordinates such as

#!/usr/bin/env python2.7
import linuxcnc

stat = linuxcnc.stat()
stat.poll()
print(stat.actual_position)

Example output:

(7.4023762662053105, 26.443582149595567, 297.289833343029, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0)

 

Posted by Uli Köhler in LinuxCNC, Python

LinuxCNC: Custom M-100 G-Code that logs time, parameters & filename to CSV

Place this file in e.g. linuxcnc/configs/myCNC/custom-mcode/M100

#!/usr/bin/env python2.7
# M100: write timestamp, parameters and gcode filename to linuxcnc/logM100.txt
import sys
import linuxcnc
from datetime import datetime
dt = datetime.now()

stat = linuxcnc.stat() # create a connection to the status channel
stat.poll()

with open("/home/cnc/linuxcnc/logM100.txt", "a") as outfile:
    outfile.write("{} | Args={} | Path={}\n".format(dt.isoformat(), ", ".join(sys.argv[1:]), stat.file))

and make executable using

chmod a+x ./linuxcnc/configs/myCNC/custom-mcodes/M100
Posted by Uli Köhler in LinuxCNC, Python

LinuxCNC: How to read current position in Python and log to CSV

Also see our post on how to just read the position especially if you don’t care about the CSV logging: LinuxCNC: How to find current position using Python

This script will log the position to CSV approximately every millisecond. The position will be logged in machine coordinates.

#!/usr/bin/env python2.7
import linuxcnc
import datetime
import time

stat = linuxcnc.stat()

with open("position-log.csv", "w") as outfile:
    while True:
       dt = datetime.datetime.now()
       stat.poll()
       x,y,z,a,b,c,u,v,w = stat.actual_position
       outfile.write("{},{:.4f},{:.4f},{:.4f}\n".format(dt.isoformat(), x, y, z))
       time.sleep(0.001)

 

Posted by Uli Köhler in LinuxCNC, Python

How to parse linuxcnc.var using Python

with open("/home/cnc/linuxcnc/configs/myCNC/linuxcnc.var") as infile:
    lines = infile.readlines()
    splitted = [line.strip().partition("\t") for line in lines]
    linuxCNCVar = {int(splitResult[0]): float(splitResult[-1]) for splitResult in splitted}

Example output for linuxCNCVar:

{5161: 0.0,
 5162: 0.0,
 5163: 0.0,
 5164: 0.0,
 5165: 0.0,
 5166: 0.0,
 5167: 0.0,
 5168: 0.0,
 5169: 0.0,
 5181: 0.0,
 5182: 0.0,
 5183: 0.0,
 5184: 0.0,
 5185: 0.0,
 5186: 0.0,
 5187: 0.0,
 5188: 0.0,
 5189: 0.0,
 5210: 0.0,
 5211: 0.0,
 5212: 0.0,
 5213: 0.0,
 5214: 0.0,
 5215: 0.0,
 5216: 0.0,
 5217: 0.0,
 5218: 0.0,
 5219: 0.0,
 5220: 1.0,
 5221: 7.40119,
 5222: 13.549249,
 5223: 298.476583,
 5224: 0.0,
 5225: 0.0,
 5226: 0.0,
 5227: 0.0,
 5228: 0.0,
 5229: 0.0,
 5230: 0.0,
 5241: 36.868548,
 5242: 141.752329,
 5243: 207.563262,
 5244: 0.0,
 5245: 0.0,
 5246: 0.0,
 5247: 0.0,
 5248: 0.0,
 5249: 0.0,
 5250: 0.0,
 5261: 36.868548,
 5262: 145.029329,
 5263: 207.563262,
 5264: 0.0,
 5265: 0.0,
 5266: 0.0,
 5267: 0.0,
 5268: 0.0,
 5269: 0.0,
 5270: 0.0,
 5281: 36.89201,
 5282: 144.834924,
 5283: 207.935105,
 5284: 0.0,
 5285: 0.0,
 5286: 0.0,
 5287: 0.0,
 5288: 0.0,
 5289: 0.0,
 5290: 0.0,
 5301: 0.0,
 5302: 0.0,
 5303: 0.0,
 5304: 0.0,
 5305: 0.0,
 5306: 0.0,
 5307: 0.0,
 5308: 0.0,
 5309: 0.0,
 5310: 0.0,
 5321: 0.0,
 5322: 0.0,
 5323: 0.0,
 5324: 0.0,
 5325: 0.0,
 5326: 0.0,
 5327: 0.0,
 5328: 0.0,
 5329: 0.0,
 5330: 0.0,
 5341: 0.0,
 5342: 0.0,
 5343: 0.0,
 5344: 0.0,
 5345: 0.0,
 5346: 0.0,
 5347: 0.0,
 5348: 0.0,
 5349: 0.0,
 5350: 0.0,
 5361: 0.0,
 5362: 0.0,
 5363: 0.0,
 5364: 0.0,
 5365: 0.0,
 5366: 0.0,
 5367: 0.0,
 5368: 0.0,
 5369: 0.0,
 5370: 0.0,
 5381: 0.0,
 5382: 0.0,
 5383: 0.0,
 5384: 0.0,
 5385: 0.0,
 5386: 0.0,
 5387: 0.0,
 5388: 0.0,
 5389: 0.0,
 5390: 0.0}

 

Posted by Uli Köhler in LinuxCNC, Python

How to fix tox AttributeError: module ‘virtualenv.create.via_global_ref.builtin.cpython.mac_os’ has no attribute ‘CPython2macOsArmFramework’

Problem:

While trying to run Python tests using tox, you see an error message like

GLOB sdist-make: /home/uli/dev/UliEngineering/setup.py
py37 create: /home/uli/dev/UliEngineering/.tox/py37
ERROR: InterpreterNotFound: python3.7
py38 create: /home/uli/dev/UliEngineering/.tox/py38
ERROR: invocation failed (exit code 1), logfile: /home/uli/dev/UliEngineering/.tox/py38/log/py38-0.log
========================================================================================= log start =========================================================================================
AttributeError: module 'virtualenv.create.via_global_ref.builtin.cpython.mac_os' has no attribute 'CPython2macOsArmFramework'

========================================================================================== log end ==========================================================================================
ERROR: InvocationError for command /usr/bin/python3 -m virtualenv --no-download --python /usr/bin/python3 py38 (exited with code 1)
__________________________________________________________________________________________ summary __________________________________________________________________________________________
ERROR:  py37: InterpreterNotFound: python3.7
ERROR:   py38: InvocationError for command /usr/bin/python3 -m virtualenv --no-download --python /usr/bin/python3 py38 (exited with code 1)

Solution:

This error occurs because two different and incompatible version of virtualenv are installed.

First, uninstall all versions using these commands:

First run as user (not as root) to uninstall any locally installed package in ~/.local:

pip3 uninstall virtualenv

Then, uninstall global packages

sudo pip3 uninstall virtualenv

Now uninstall the apt package on Debian/Ubuntu (uninstall using your package manager for other distros):

sudo apt purge python3-virtualenv

Now it’s time to install one version of virtualenv:

sudo pip3 install virtualenv

After that, you can try running tox again.

Posted by Uli Köhler in Python

Create a numpy array of one datetime64 per day and mark the start of the month

startdate = np.datetime64('2022-01-01T00:00:00.000000')
 # 10000 days. This array contains 0 (Day 0), 1 (Day 1), etc
day_offsets = np.arange(10000, dtype=np.int64)
usec_per_day = int(1e6) * 86400 # 86.4k sec per day, 1e6 microseconds per second
# Compute microseconds offset from first day
usec_offsets = day_offsets * usec_per_day
# Compute timestamps
timestamps = startdate + usec_offsets

# Mark start of month in boolean array
is_start_of_month = np.zeros_like(timestamps, dtype=bool)
for index, timestamp in np.ndenumerate(timestamps):
    is_start_of_month[index[0]] = timestamp.astype(datetime).day == 1

Using this method, timestamps will be the start of each day

array(['2022-01-01T00:00:00.000000', '2022-01-02T00:00:00.000000',
       '2022-01-03T00:00:00.000000', ..., '2101-12-30T00:00:00.000000',
       '2101-12-31T00:00:00.000000', '2102-01-01T00:00:00.000000'],
      dtype='datetime64[us]')

 

Posted by Uli Köhler in Data science, Python

Pandas: How to apply numpy function to every column

You can use df.transform(func, axis=0) to apply a numpy function. This leverages the fact that numpy functions work with pandas Series objects.

Example based on How to create pandas time series DataFrame example dataset:

# Load pre-built time series example dataset
df = pd.read_csv("https://datasets.techoverflow.net/timeseries-example.csv", parse_dates=["Timestamp"])
df.set_index("Timestamp", inplace=True)

# np.square will be called individually for each column
new_df = df.transform(np.square, axis=0)

Output

Original time series:

Squared time series:

Full example code

import numpy as np
import pandas as pd
from matplotlib import pyplot as plt

# Load pre-built time series example dataset
df = pd.read_csv("https://datasets.techoverflow.net/timeseries-example.csv", parse_dates=["Timestamp"])
df.set_index("Timestamp", inplace=True)

# np.sqrt will be called individually for each column
new_df = df.transform(np.square, axis=0)

# Plot subsection of original DF for better visibility
df.iloc[:len(df)//2].plot()
plt.gcf().set_size_inches(10,5)
plt.savefig("Normal-Timeseries.svg")

# Plot subsection of transformed DF for better visibility
new_df.iloc[:len(df)//2].plot()
plt.gcf().set_size_inches(10,5)
plt.savefig("Square-Timeseries.svg")

 

Posted by Uli Köhler in pandas, Python

Numpy nth root: How to

To find the nth root of number or array x in numpy use:

np.power(x, (1/n))

n can be a natural number like 4 but it can also be a floating point number like 3.26.

Example: Compute the 78th root of 1.234

>>> import numpy as np
>>> np.power(1.234, (1/78))
1.0026992894299456

 

Posted by Uli Köhler in Python

How to iterate all ISO3166-Alpha2 codes in Python

First install the iso3166 library:

pip install iso3166

Now you can iterate all the alpha2 codes like DE and US like this:

from iso3166 import countries

for alpha2 in [country.alpha2 for country in countries]:
    print(country)

This will print:

AF
AX
AL
DZ
AS
AD
AO
AI
AQ
AG
AR
AM
AW
AU
AT
AZ
BS
BH
BD
BB
BY
BE
BZ
BJ
BM
BT
BO
BQ
BA
BW
BV
BR
IO
BN
BG
BF
BI
KH
CM
CA
CV
KY
CF
TD
CL
CN
CX
CC
CO
KM
CG
CD
CK
CR
CI
HR
CU
CW
CY
CZ
DK
DJ
DM
DO
EC
EG
SV
GQ
ER
EE
ET
FK
FO
FJ
FI
FR
GF
PF
TF
GA
GM
GE
DE
GH
GI
GR
GL
GD
GP
GU
GT
GG
GN
GW
GY
HT
HM
VA
HN
HK
HU
IS
IN
ID
IR
IQ
IE
IM
IL
IT
JM
JP
JE
JO
KZ
KE
KI
KP
KR
XK
KW
KG
LA
LV
LB
LS
LR
LY
LI
LT
LU
MO
MK
MG
MW
MY
MV
ML
MT
MH
MQ
MR
MU
YT
MX
FM
MD
MC
MN
ME
MS
MA
MZ
MM
NA
NR
NP
NL
NC
NZ
NI
NE
NG
NU
NF
MP
NO
OM
PK
PW
PS
PA
PG
PY
PE
PH
PN
PL
PT
PR
QA
RE
RO
RU
RW
BL
SH
KN
LC
MF
PM
VC
WS
SM
ST
SA
SN
RS
SC
SL
SG
SX
SK
SI
SB
SO
ZA
GS
SS
ES
LK
SD
SR
SJ
SZ
SE
CH
SY
TW
TJ
TZ
TH
TL
TG
TK
TO
TT
TN
TR
TM
TC
TV
UG
UA
AE
GB
US
UM
UY
UZ
VU
VE
VN
VG
VI
WF
EH
YE
ZM
ZW

 

Posted by Uli Köhler in Python

matplotlib: How to easily format y value as percent [%]

When plotting our time series example dataset, this is the resulting plot

This post shows how to easily plot this dataset with an y axis formatted as percent. We will assume that 1.00 maps to 100%. This post is based on our previous work on Matplotlib custom SI-prefix unit tick formatter:

Note that for pandas, you need to first call df.plot() and call set_major_formatter() after that!

import matplotlib.ticker as mtick
df.plot()
plt.gca().yaxis.set_major_formatter(mtick.PercentFormatter(xmax=1.0))

If you instead want 100.0 to map to 100%, just use xmax=100.0:

import matplotlib.ticker as mtick
df.plot()
plt.gca().yaxis.set_major_formatter(mtick.PercentFormatter(xmax=1.0))

Full example

import matplotlib.ticker as mtick

# Load pre-built time series example dataset
df = pd.read_csv("https://datasets.techoverflow.net/timeseries-example.csv", parse_dates=["Timestamp"])
df.set_index("Timestamp", inplace=True)

# Plot with Y axis scaled as percent
df.plot()
plt.gca().yaxis.set_major_formatter(mtick.PercentFormatter(xmax=1.0))

 

Posted by Uli Köhler in pandas, Python

How to get first column name of pandas DataFrame

df.columns[0]

will return the name of the first column as str.

Posted by Uli Köhler in pandas, Python

How to plot multiple pandas DataFrames in a single graph

If you use code like

df1.plot()
df2.plot()

you will see both DataFrames being plotted each in their separate graphs/plots.

In order to plot both of them in a single plot, use

ax = df1.plot()
df2.plot(ax=ax)
Posted by Uli Köhler in pandas, Python

How to combine two pandas DataFrames with the same index

If you have two pandas DataFrames you want to join where the index in both DataFrames are and you want to obtain a DataFrame where the respective columns are set to NaN if there is no value from the respective DataFrame, this is typically the correct way to do it:

df_compare = pd.concat([df1, df2], axis=1, join='outer')

If you only want to keep values where both DataFrames have some value, use join='outer'

Posted by Uli Köhler in pandas, Python

How to rename a single column of a pandas DataFrame

In our previous post How to replace string in column names in Pandas DataFrame we already covered a generic method of renaming pandas columns by replacing strings.

If you just want to rename a specific column, say, the column is named value and you want to convert it to , use this snippet:

new_df = df.rename(columns=lambda s: "€" if s == "value" else s)

or use inplace=True if you want to modify the original DataFrame:

df.rename(columns=lambda s: "€" if s == "value" else s, inplace=True)

 

 

Posted by Uli Köhler in pandas, Python