Python

How to create temporary directory in Python

Minimal example:

import tempfile
tempdir = tempfile.mkdtemp()
print(tempdir) # prints e.g. /tmp/tmpvw0936nd

tempfile.mkdtemp() will automatically create that directory. The directory will not automatically be deleted!

Custom prefix (recommended):

import tempfile
tempdir = tempfile.mkdtemp(prefix="myapplication-")
print(tempdir) # prints e.g. /tmp/myapplication-ztcy6s2w

 

Posted by Uli Köhler in Python

How to disable InsecureRequestWarning: Unverified HTTPS request is being made.

If you use requests or urllib3, requests with SSL verification disabled will print this warning:

/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)

If you want to disable this warning and you can’t just enable verification, add this code

import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

at the top of your Python file.

Posted by Uli Köhler in Python

Python subprocess.check_output(): Set working directory

If you have code that uses subprocess.check_output() to call a command like

subprocess.check_output("ls .", shell=True)

you can use the cwd=... argument of subprocess.check_output() to define the working directory. Example:

subprocess.check_output("ls .", cwd="../", shell=True)

cwd means change working directory and is interpreted relative to the current working directory. However, you can also use absolute paths:

subprocess.check_output("ls .", cwd="/etc/", shell=True)

 

Posted by Uli Köhler in Python

Python bottle minimal redirect example

To redirect in bottle, use this snippet:

response.status = 303
response.set_header('Location', 'https://techoverflow.net')

303 is the HTTP response code See Other. In certain applications you might want to use 301 (permanent redirect) or 307 (temporary redirect) instead.

Full example:

#!/usr/bin/env python3
from bottle import route, run, response

@route('/')
def redirect():
    response.status = 303
    response.set_header('Location', 'https://techoverflow.net')

run(host='localhost', port=9000)

 

Posted by Uli Köhler in Python

Python bottle minimal example

Minimal hello world-like example using the bottle HTTP server:

#!/usr/bin/env python3
from bottle import route, run

@route('/')
def index():
    return "Hello bottle!"

run(host='localhost', port=9000)

Run this file (and keep it running) and then navigate to http://localhost:9000 to see the Hello bottle! message.

In case you see this error message:

Traceback (most recent call last):
  File "./server.py", line 2, in <module>
    from bottle import route, run
ModuleNotFoundError: No module named 'bottle'

you need to install bottle using

sudo pip3 install bottle

or

sudo pip install bottle

depending on which Python version and configuration you use.

Posted by Uli Köhler in Python

How to list available projections in pyproj

To get a list of all available projections for pyproj, use this code:

import pyproj
print(pyproj.pj_list)

For pyproj-2.2.0 on my Ubuntu 18.04 computer, this is the resulting list:

{
    'aea': 'Albers Equal Area',
    'aeqd': 'Azimuthal Equidistant',
    'affine': 'Affine transformation',
    'airy': 'Airy',
    'aitoff': 'Aitoff',
    'alsk': 'Mod. Stereographic of Alaska',
    'apian': 'Apian Globular I',
    'august': 'August Epicycloidal',
    'axisswap': 'Axis ordering',
    'bacon': 'Bacon Globular',
    'bertin1953': 'Bertin 1953',
    'bipc': 'Bipolar conic of western hemisphere',
    'boggs': 'Boggs Eumorphic',
    'bonne': 'Bonne (Werner lat_1=90)',
    'calcofi': 'Cal Coop Ocean Fish Invest Lines/Stations',
    'cart': 'Geodetic/cartesian conversions',
    'cass': 'Cassini',
    'cc': 'Central Cylindrical',
    'ccon': 'Central Conic',
    'cea': 'Equal Area Cylindrical',
    'chamb': 'Chamberlin Trimetric',
    'collg': 'Collignon',
    'comill': 'Compact Miller',
    'crast': 'Craster Parabolic (Putnins P4)',
    'deformation': 'Kinematic grid shift',
    'denoy': 'Denoyer Semi-Elliptical',
    'eck1': 'Eckert I',
    'eck2': 'Eckert II',
    'eck3': 'Eckert III',
    'eck4': 'Eckert IV',
    'eck5': 'Eckert V',
    'eck6': 'Eckert VI',
    'eqearth': 'Equal Earth',
    'eqc': 'Equidistant Cylindrical (Plate Carree)',
    'eqdc': 'Equidistant Conic',
    'euler': 'Euler',
    'etmerc': 'Extended Transverse Mercator',
    'fahey': 'Fahey',
    'fouc': 'Foucaut',
    'fouc_s': 'Foucaut Sinusoidal',
    'gall': 'Gall (Gall Stereographic)',
    'geoc': 'Geocentric Latitude',
    'geocent': 'Geocentric',
    'geogoffset': 'Geographic Offset',
    'geos': 'Geostationary Satellite View',
    'gins8': 'Ginsburg VIII (TsNIIGAiK)',
    'gn_sinu': 'General Sinusoidal Series',
    'gnom': 'Gnomonic',
    'goode': 'Goode Homolosine',
    'gs48': 'Mod. Stereographic of 48 U.S.',
    'gs50': 'Mod. Stereographic of 50 U.S.',
    'hammer': 'Hammer & Eckert-Greifendorff',
    'hatano': 'Hatano Asymmetrical Equal Area',
    'healpix': 'HEAPJ_LPix',
    'rhealpix': 'rHEAPJ_LPix',
    'helmert': '3(6)-, 4(8)- and 7(14)-parameter Helmert shift',
    'hgridshift': 'Horizontal grid shift',
    'horner': 'Horner polynomial evaluation',
    'igh': 'Interrupted Goode Homolosine',
    'imw_p': 'International Map of the World Polyconic',
    'isea': 'Icosahedral Snyder Equal Area',
    'kav5': 'Kavraisky V',
    'kav7': 'Kavraisky VII',
    'krovak': 'Krovak',
    'labrd': 'Laborde',
    'laea': 'Lambert Azimuthal Equal Area',
    'lagrng': 'Lagrange',
    'larr': 'Larrivee',
    'lask': 'Laskowski',
    'lonlat': 'Lat/long (Geodetic)',
    'latlon': 'Lat/long (Geodetic alias)',
    'latlong': 'Lat/long (Geodetic alias)',
    'longlat': 'Lat/long (Geodetic alias)',
    'lcc': 'Lambert Conformal Conic',
    'lcca': 'Lambert Conformal Conic Alternative',
    'leac': 'Lambert Equal Area Conic',
    'lee_os': 'Lee Oblated Stereographic',
    'loxim': 'Loximuthal',
    'lsat': 'Space oblique for LANDSAT',
    'mbt_s': 'McBryde-Thomas Flat-Polar Sine (No. 1)',
    'mbt_fps': 'McBryde-Thomas Flat-Pole Sine (No. 2)',
    'mbtfpp': 'McBride-Thomas Flat-Polar Parabolic',
    'mbtfpq': 'McBryde-Thomas Flat-Polar Quartic',
    'mbtfps': 'McBryde-Thomas Flat-Polar Sinusoidal',
    'merc': 'Mercator',
    'mil_os': 'Miller Oblated Stereographic',
    'mill': 'Miller Cylindrical',
    'misrsom': 'Space oblique for MISR',
    'moll': 'Mollweide',
    'molobadekas': 'Molodensky-Badekas transformation',
    'molodensky': 'Molodensky transform',
    'murd1': 'Murdoch I',
    'murd2': 'Murdoch II',
    'murd3': 'Murdoch III',
    'natearth': 'Natural Earth',
    'natearth2': 'Natural Earth 2',
    'nell': 'Nell',
    'nell_h': 'Nell-Hammer',
    'nicol': 'Nicolosi Globular',
    'nsper': 'Near-sided perspective',
    'nzmg': 'New Zealand Map Grid',
    'noop': 'No operation',
    'ob_tran': 'General Oblique Transformation',
    'ocea': 'Oblique Cylindrical Equal Area',
    'oea': 'Oblated Equal Area',
    'omerc': 'Oblique Mercator',
    'ortel': 'Ortelius Oval',
    'ortho': 'Orthographic',
    'pconic': 'Perspective Conic',
    'patterson': 'Patterson Cylindrical',
    'pipeline': 'Transformation pipeline manager',
    'poly': 'Polyconic (American)',
    'pop': 'Retrieve coordinate value from pipeline stack',
    'push': 'Save coordinate value on pipeline stack',
    'putp1': 'Putnins P1',
    'putp2': 'Putnins P2',
    'putp3': 'Putnins P3',
    'putp3p': "Putnins P3'",
    'putp4p': "Putnins P4'",
    'putp5': 'Putnins P5',
    'putp5p': "Putnins P5'",
    'putp6': 'Putnins P6',
    'putp6p': "Putnins P6'",
    'qua_aut': 'Quartic Authalic',
    'qsc': 'Quadrilateralized Spherical Cube',
    'robin': 'Robinson',
    'rouss': 'Roussilhe Stereographic',
    'rpoly': 'Rectangular Polyconic',
    'sch': 'Spherical Cross-track Height',
    'sinu': 'Sinusoidal (Sanson-Flamsteed)',
    'somerc': 'Swiss. Obl. Mercator',
    'stere': 'Stereographic',
    'sterea': 'Oblique Stereographic Alternative',
    'gstmerc': 'Gauss-Schreiber Transverse Mercator (aka Gauss-Laborde Reunion)',
    'tcc': 'Transverse Central Cylindrical',
    'tcea': 'Transverse Cylindrical Equal Area',
    'times': 'Times',
    'tissot': 'Tissot',
    'tmerc': 'Transverse Mercator',
    'tobmerc': 'Tobler-Mercator',
    'tpeqd': 'Two Point Equidistant',
    'tpers': 'Tilted perspective',
    'unitconvert': 'Unit conversion',
    'ups': 'Universal Polar Stereographic',
    'urm5': 'Urmaev V',
    'urmfps': 'Urmaev Flat-Polar Sinusoidal',
    'utm': 'Universal Transverse Mercator (UTM)',
    'vandg': 'van der Grinten (I)',
    'vandg2': 'van der Grinten II',
    'vandg3': 'van der Grinten III',
    'vandg4': 'van der Grinten IV',
    'vitk1': 'Vitkovsky I',
    'vgridshift': 'Vertical grid shift',
    'wag1': 'Wagner I (Kavraisky VI)',
    'wag2': 'Wagner II',
    'wag3': 'Wagner III',
    'wag4': 'Wagner IV',
    'wag5': 'Wagner V',
    'wag6': 'Wagner VI',
    'wag7': 'Wagner VII',
    'webmerc': 'Web Mercator / Pseudo Mercator',
    'weren': 'Werenskiold I',
    'wink1': 'Winkel I',
    'wink2': 'Winkel II',
    'wintri': 'Winkel Tripel'
}

 

Posted by Uli Köhler in Python

Python threading.Thread minimal example

This example shows how to subclass and start a threading.Thread:

#!/usr/bin/env python3
import threading

class MyThread(threading.Thread):
    def run(self):
        print("My thread!")

my_thread = MyThread()
my_thread.start()
print("Main thread")

This will print

My thread!
Main thread

(it might also print Main thread first, depending on what gets executed first)

Posted by Uli Köhler in Python

How to get directory of current Python script

import os 
script_directory = os.path.dirname(os.path.realpath(__file__))

This will always get the directory of the .py file running that code, so if you have .py files in a subdirectory/module this might return that subdirectory.

Original (partial) source for the solution: StackOverflow

Posted by Uli Köhler in Python

How to fix Python ‘ImportError: No module named wx’ on Ubuntu

Problem:

You want to run a Python script, e.g. Pronterface, but you see an error message like this:

wxPython is not installed. This program requires wxPython to run.
Traceback (most recent call last):
  File "./pronterface.py", line 23, in <module>
    import wx  # NOQA
ImportError: No module named wx

Solution:

sudo apt install python-wxtools

 

Posted by Uli Köhler in Python

How to parse ISO8601 datetime in Python

The easiest way to parse a ISO8601 timestamp like 2019-05-29T19:50:55.463+02:00 is to install the maya library using e.g.

sudo pip3 install maya

In case you still use Python2, use pip instead of pip3.

After installing maya, use it like this:

import maya
result = maya.parse('2019-05-29T19:50:55.463+02:00').datetime()
# Result is a standard datetime object!
print(result)

This will print

2019-05-29 17:50:55.463000+00:00

Notice how the original timestamp from timezone +02:00 is automatically converted to a UTC datetime object.

maya can parse a lot of different formats e.g.

print(maya.parse('2019-05-29T19:50:55+02:00').datetime()) # No fractional seconds
print(maya.parse('2019-05-29T19:50:55').datetime()) # No timezone (UTC assumed)
print(maya.parse('2019-05-29 19:50:55').datetime()) # Not ISO8601
print(maya.parse('2019-05-29').datetime()) # 00:00:00 & UTC assumed

 

Posted by Uli Köhler in Python

What’s the formula for 2/3/4 parallel resistors?

Formula for 2 parallel resistors R1 and R2:

R_{total} = \frac{1}{\frac{1}{R_1} + \frac{1}{R_2}}

Python code:

rtotal = 1. / (1./r1 + 1./r2)

Formula for 3 parallel resistors R1 and R2:

R_{total} = \frac{1}{\frac{1}{R_1} + \frac{1}{R_2} + \frac{1}{R_3}}

Python code:

rtotal = 1. / (1./r1 + 1./r2 + 1./r3)

Formula for 4 parallel resistors R1 and R2:

R_{total} = \frac{1}{\frac{1}{R_1} + \frac{1}{R_2} + \frac{1}{R_3} + \frac{1}{R_4}}

Python code:

rtotal = 1. / (1./r1 + 1./r2 + 1./r3 + 1./r4)

 

Posted by Uli Köhler in Electronics, Python

How to fix numpy TypeError: Cannot cast ufunc subtract output from dtype(‘float64’) to dtype(‘int64’) with casting rule ‘same_kind’

Problem:

You are trying to do a simple arithmetic operation on a NumPy array but you see an error message like

TypeError: Cannot cast ufunc subtract output from dtype('float64') to dtype('int64') with casting rule 'same_kind'

Solution:

You are trying to substract a float from an int64 array. This does not work with operators like += or -=

Example:

import numpy as np

data = np.asarray([1, 2, 3, 4], dtype=np.int64) # This is an int array!

print(data - 5) # This works
print(data - 5.0) # This works as well
# This raises: Cannot cast ufunc subtract output from dtype('float64') to dtype('int64') with casting rule 'same_kind'
data -= 5.0

Option 1 (preferred):

Use - instead of -=: Instead of data -= 5.0 use data = data - 5.0

Option 2:

Explicitly cast data to float (or the first dtype of your error message):

data = data.astype('float64')
# Now this works
data -= 5.0

This option is not preferred since doing it requires using the correct datatype. The first option works without regarding the actual datatype.

Posted by Uli Köhler in Python

How to iterate all days of year using Python

To iterate all days in a year using Python you can use this function snippet:

from collections import namedtuple
from calendar import monthrange

Date = namedtuple("Date", ["year", "month", "day"])

def all_dates_in_year(year=2019):
    for month in range(1, 13): # Month is always 1..12
        for day in range(1, monthrange(year, month)[1] + 1):
            yield Date(year, month, day)

Usage example:

for date in all_dates_in_year(2019):
    print(date)

This will print (excerpt):

Date(year=2019, month=1, day=1)
Date(year=2019, month=1, day=2)
Date(year=2019, month=1, day=3)
[...]
Date(year=2019, month=2, day=27)
Date(year=2019, month=2, day=28)
Date(year=2019, month=3, day=1)
[...]
Date(year=2019, month=12, day=29)
Date(year=2019, month=12, day=30)
Date(year=2019, month=12, day=31)

You can also install my Python3 library UliEngineering using sudo pip3 install -U UliEngineering and then use all_dates_in_year() by importing it like this:

from UliEngineering.Utils.Date import *

 

Also see How to get number of days in month in Python

 

Posted by Uli Köhler in Python

How to get number of days in month in Python

If you want to find out how many days a month using Python use this snippet:

from calendar import monthrange

num_days = monthrange(2019, 2)[1] # num_days = 28

print(num_days) # Prints 28

The calendar module automatically takes into account leap years.

You can also use this function snippet:

from calendar import monthrange

def number_of_days_in_month(year=2019, month=2):
    return monthrange(year, month)[1]

# Usage example:
print(number_of_days_in_month(2019, 2)) # Prints 28

You can also install my Python3 library UliEngineering using sudo pip3 install -U UliEngineering and then use number_of_days_in_month() by importing it like this:

from UliEngineering.Utils.Date import *

 

Posted by Uli Köhler in Python

How to get unit/resolution of NumPy np.datetime64 object

NumPy’s datetime64 objects are represented as 64 bit integers (that’s what the 64 in the name means).

In order to find out what the resolution (e.g. us, ns etc) first install my Python 3 UliEngineering library using

sudo pip3 install -U UliEngineering

and then use datetime64_unit() from UliEngineering.Utils.NumPy like this:

from UliEngineering.Utils.NumPy import *

my_datetime = datetime64_now()
print(datetime64_resolution(my_datetime)) # Prints "us"
Posted by Uli Köhler in Python

How to get unit/resolution of NumPy np.timedelta64 object

In order to get the unit (e.g. us, ms, ns, …) of a np.timedelta64 object like

my_timedelta = np.timedelta64(625, 'us') # Unit is 'us'

first install my Python 3 UliEngineering library using

sudo pip3 install -U UliEngineering

and then use timedelta64_unit() from UliEngineering.Utils.NumPy like this:

import numpy as np
from UliEngineering.Utils.NumPy import *

my_timedelta = np.timedelta64(625, 'us')
print(timedelta64_resolution(my_timedelta)) # Prints "us"
Posted by Uli Köhler in Python

How to convert numpy timedelta (np.timedelta64) object to integer

If you have a NumPy np.timedelta64 object like

import numpy as np
my_timedelta = np.timedelta64(625, 'us')

a common task is to convert it to an integer.

This is often as easy as

my_timedelta.astype(int) # = 625, type: np.int64

This will give you the number (which is always an integer!) stored in the np.timedelta64 object, however it will ignore the unit  (e.g. us i.e. microseconds in our example).

To get the unit of the timedelta, first install UliEngineering using sudo pip3 install -U UliEngineering which you can then use like this:

import numpy as np
from UliEngineering.Utils.NumPy import *

my_timedelta = np.timedelta64(625, 'us')
print(timedelta64_resolution(my_timedelta)) # Prints "us"
Posted by Uli Köhler in Python

How to create a new numpy timedelta (np.timedelta64) object

There are two basic ways to create a new np.timedelta64 object:

Construct directly:

import numpy as np
my_timedelta = np.timedelta64(625, 'us')

Note that the constructor will only work with ints, i.e. np.timedelta64(625.5, 'us') won’t work. See

Construct as a difference between two np.datetime64 objects:

See How to get current datetime as numpy datetime (np.datetime64) for more details on how to get the current datetime as np.datetime64.

For any two np.datetime64 instances, we can compute the difference using

from datetime import datetime
import numpy as np

# There will be a minimal time difference between those two
dt1 = np.datetime64(datetime.now())
dt2 = np.datetime64(datetime.now())

my_timedelta = dt2 - dt1
print(my_timedelta) # For me, this prints "16 microseconds"
Posted by Uli Köhler in Python

How to get current datetime as numpy datetime (np.datetime64)

The simplest way of getting the current time as np.datetime64 object is to combine datetime.datetime.now() and the np.datetime64() constructor:

from datetime import datetime
import numpy as np

dt_now = np.datetime64(datetime.now())

print(dt_now)

Or use this function for convenience:

from datetime import datetime
import numpy as np

def datetime64_now():
    return np.datetime64(datetime.now())

dt_now = datetime64_now()
print(dt_now)

datetime64_now() is also available on my library UliEngineering.
Install using sudo pip3 install -U UliEngineering, then use like this:

from UliEngineering.Utils.NumPy import datetime64_now

dt_now = datetime64_now()
print(dt_now)
Posted by Uli Köhler in Python

How to list attributes of a Python object without __dict__

Usually you can list all attributes and functions of a Python object using __dir__:

from collections import namedtuple
nt = namedtuple("Foo", [])
print(nt.__dict__) # Prints attributes and functions of nt.

This can be very useful to find out e.g. which functions you can call on an object:

However for some objects like datetime.datetime this doesn’t work. Attempting to run

from datetime import datetime
dt = datetime.now()
print(dt.__dict__)

will result in

Traceback (most recent call last):
  File "test.py", line 3, in <module>
AttributeError: 'datetime.datetime' object has no attribute '__dict__'

So how can you find out which attributes your datetime.datetime object has and what function you can call on it?

Use dir():

from datetime import datetime
dt = datetime.now()
print(dir(dt))

This will print e.g.

['__add__', '__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__ne__', '__new__', '__radd__', '__reduce__', '__reduce_ex__', '__repr__', '__rsub__', '__setattr__', '__sizeof__', '__str__', '__sub__', '__subclasshook__', 'astimezone', 'combine', 'ctime', 'date', 'day', 'dst', 'fold', 'fromordinal', 'fromtimestamp', 'hour', 'isocalendar', 'isoformat', 'isoweekday', 'max', 'microsecond', 'min', 'minute', 'month', 'now', 'replace', 'resolution', 'second', 'strftime', 'strptime', 'time', 'timestamp', 'timetuple', 'timetz', 'today', 'toordinal', 'tzinfo', 'tzname', 'utcfromtimestamp', 'utcnow', 'utcoffset', 'utctimetuple', 'weekday', 'year']

 

Posted by Uli Köhler in Python