Python

If you have a NumPy np.timedelta64 object like

import numpy as np
my_timedelta = np.timedelta64(625, 'us')

a common task is to convert it to an integer.

This is often as easy as

my_timedelta.astype(int) # = 625, type: np.int64

This will give you the number (which is always an integer!) stored in the np.timedelta64 object, however it will ignore the unit  (e.g. us i.e. microseconds in our example).

To get the unit of the timedelta, first install UliEngineering using sudo pip3 install -U UliEngineering which you can then use like this:

import numpy as np
from UliEngineering.Utils.NumPy import *

my_timedelta = np.timedelta64(625, 'us')
print(timedelta64_resolution(my_timedelta)) # Prints "us"

There are two basic ways to create a new np.timedelta64 object:

Construct directly:

import numpy as np
my_timedelta = np.timedelta64(625, 'us')

Note that the constructor will only work with ints, i.e. np.timedelta64(625.5, 'us') won’t work. See

Construct as a difference between two np.datetime64 objects:

See How to get current datetime as numpy datetime (np.datetime64) for more details on how to get the current datetime as np.datetime64.

For any two np.datetime64 instances, we can compute the difference using

from datetime import datetime
import numpy as np

# There will be a minimal time difference between those two
dt1 = np.datetime64(datetime.now())
dt2 = np.datetime64(datetime.now())

my_timedelta = dt2 - dt1
print(my_timedelta) # For me, this prints "16 microseconds"

The simplest way of getting the current time as np.datetime64 object is to combine datetime.datetime.now() and the np.datetime64() constructor:

from datetime import datetime
import numpy as np

dt_now = np.datetime64(datetime.now())

print(dt_now)

Or use this function for convenience:

from datetime import datetime
import numpy as np

def datetime64_now():
    return np.datetime64(datetime.now())

dt_now = datetime64_now()
print(dt_now)

datetime64_now() is also available on my library UliEngineering.
Install using sudo pip3 install -U UliEngineering, then use like this:

from UliEngineering.Utils.NumPy import datetime64_now

dt_now = datetime64_now()
print(dt_now)

Usually you can list all attributes and functions of a Python object using __dir__:

from collections import namedtuple
nt = namedtuple("Foo", [])
print(nt.__dict__) # Prints attributes and functions of nt.

This can be very useful to find out e.g. which functions you can call on an object:

However for some objects like datetime.datetime this doesn’t work. Attempting to run

from datetime import datetime
dt = datetime.now()
print(dt.__dict__)

will result in

Traceback (most recent call last):
  File "test.py", line 3, in <module>
AttributeError: 'datetime.datetime' object has no attribute '__dict__'

So how can you find out which attributes your datetime.datetime object has and what function you can call on it?

Use dir():

from datetime import datetime
dt = datetime.now()
print(dir(dt))

This will print e.g.

['__add__', '__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__ne__', '__new__', '__radd__', '__reduce__', '__reduce_ex__', '__repr__', '__rsub__', '__setattr__', '__sizeof__', '__str__', '__sub__', '__subclasshook__', 'astimezone', 'combine', 'ctime', 'date', 'day', 'dst', 'fold', 'fromordinal', 'fromtimestamp', 'hour', 'isocalendar', 'isoformat', 'isoweekday', 'max', 'microsecond', 'min', 'minute', 'month', 'now', 'replace', 'resolution', 'second', 'strftime', 'strptime', 'time', 'timestamp', 'timetuple', 'timetz', 'today', 'toordinal', 'tzinfo', 'tzname', 'utcfromtimestamp', 'utcnow', 'utcoffset', 'utctimetuple', 'weekday', 'year']

Minimal example with one positional parameter (param):

#!/usr/bin/env python3
import argparse

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("param")
    args = parser.parse_args()
    # Your code goes here!
    print(args.param)

Minimal example with one argument (-p, --param):

#!/usr/bin/env python3
import argparse

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("-p", "--param")
    args = parser.parse_args()
    # Your code goes here!
    print(args.param)

Here’s a minimal example for a custom class that is iterable similar to a generator:

class MyIterable(object):
    def __init__(self):
        pass # ... Your code goes here!

    def __iter__(self):
        # Usually there is no need to modify this!
        return self

    def __next__(self):
        # To indicate no further values, use: raise StopIteration
        return "test" # Return the next value here. Return n

Usage example:

it = MyIterable()
print(next(it)) # Prints "test"
print(next(it)) # Prints "test"

In case you want to read more about how to create your own iterators, I recommend this tutorial.

In this example, we’ll use the UliEngineering library to convert between the three most commonly used units of temperature: Celsius (°C), Fahrenheit (°F) and Kelvin (K)

In order to install UliEngineering (a Python 3 library) run:

sudo pip3 install -U UliEngineering

We can now use the following functions from the UliEngineering.Physics.Temperature package to convert between the units of temperature:

• celsius_to_kelvin(temperature)
• kelvin_to_celsius(temperature)
• fahrenheit_to_kelvin(temperature)
• fahrenheit_to_celsius(temperature)

Tip: You can pass both numbers (like 120) or strings (like 120 °C or 0.01 °F) to most UliEngineering functions. SI prefixes like k and m are automatically decoded.

Example:

from UliEngineering.EngineerIO import auto_print
from UliEngineering.Physics.Temperature import *

# Convert to celsius and store in variable
temp = fahrenheit_to_celsius("120 °F") # temp = 48.88888888888897

# Automatically format value & print. Prints "48.9 °C"
auto_print(fahrenheit_to_celsius, "120 °F")

Additionally, UliEngineering provides normalize_temperature_celsius(temp, default_unit="°C") which takes a string and automatically recognizes the unit (if there is no unit given, the default_unit is assumed).

Examples:

• normalize_temperature_celsius("120 °C") == 120.0
• normalize_temperature_celsius("120") == 120.0
• normalize_temperature_celsius(120) == 120.0
• normalize_temperature_celsius("120 °F") == 48.88888888888897
• normalize_temperature_celsius("120 K") == -153.14999999999998
• auto_print(normalize_temperature_celsius, "120 K") prints -153.15 °C

Note that while °K is recognized by UliEngineering‘s functions, K is correctly used without a degree symbol.

Also there is normalize_temperature(temp, default_unit="°C") which is equivalent to normalize_temperature_celsius() except it returns the temperature in Kelvin instead of degrees Celsius.

Problem:

You want to test a Python package e.g. using python setup.py test.  However, you see the following error message:

usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

error: option --rednose not recognized

Solution:

Rednose is a Python package that you need to install:

sudo pip3 install rednose

or, if you don’t have pip3, try

sudo pip install rednose

Assuming timestamp is your timestamp of type google.protobuf.timestamp_pb2.Timestamp, use this snippet to convert to datetime:

from datetime import datetime

timestamp_dt = datetime.fromtimestamp(timestamp.seconds + timestamp.nanos/1e9)

Problem:

You are running a Python HTTP server using bottle. When you access your HTTP server endpoint, you see a HTTP 500 error message like

Unsupported response type: <class 'dict'>

Solution:

This occurs because you are trying to return a Python list of dictionaries, for example in

from bottle import route, run, template, response

@route('/')
def index():
    # We expect bottle to return a JSON here
    # but that doesn't happen!
    return [{"a": "b"}]

run(host='localhost', port=8080)

In order to work around this behaviour, you need to set response.content_type and explicitly use json.dumps() to convert your JSON into a string:

from bottle import route, run, template, response
import json

@route('/')
def index():
    response.content_type = 'application/json'
    return json.dumps([{"a": "b"}])

run(host='localhost', port=8080)

Use this Python script to create a draft email on your IMAP server. The email is not sent automatically but only stored in your draft folder.

#!/usr/bin/env python3
import imaplib
import ssl
import email.message
import email.charset
import time

tls_context = ssl.create_default_context()

server = imaplib.IMAP4('imap.mydomain.com')
server.starttls(ssl_context=tls_context)
server.login('email@mydomain.com', 'password')
# Select mailbox
server.select("INBOX.Drafts")
# Create message
new_message = email.message.Message()
new_message["From"] = "Your name <sender@mydomain.com>"
new_message["To"] = "Name of Recipient <recpient@mydomain.com>"
new_message["Subject"] = "Your subject"
new_message.set_payload("""
This is your message.
It can have multiple lines and
contain special characters: äöü.
""")
# Fix special characters by setting the same encoding we'll use later to encode the message
new_message.set_charset(email.charset.Charset("utf-8"))
encoded_message = str(new_message).encode("utf-8")
server.append('INBOX.Drafts', '', imaplib.Time2Internaldate(time.time()), encoded_message)
# Cleanup
server.close()

Also see Minimal Python IMAP over TLS example

Note: Under some circumstances you might want to consider using IMAP over SSL instead. See Minimal Python IMAP over SSL example

This example code will login to the server, start a TLS session, list the mailboxes and logout immediately.

#!/usr/bin/env python3
import imaplib
import ssl

# Load system's trusted SSL certificates
tls_context = ssl.create_default_context()

# Connect (unencrypted at first)
server = imaplib.IMAP4('imap.mydomain.com')
# Start TLS encryption. Will fail if TLS session can't be established
server.starttls(ssl_context=tls_context)
# Login. ONLY DO THIS AFTER server.starttls() !!
server.login('email@mydomain.com', 'password')
# Print list of mailboxes on server
code, mailboxes = server.list()
for mailbox in mailboxes:
    print(mailbox.decode("utf-8"))
# Select mailbox
server.select("INBOX")
# Cleanup
server.close()

Remember to replace:

• imap.mydomain.com with the domain name or IP address of your IMAP server
• email@mydomain.com by the email address you want to login with
• password by the password you want to login with

You need to absolutely ensure that you run server.starttls(...) first and only afterwards do server.login(...). If you fail to do so, eavesdroppers might be able to read your username and password which is not encrypted!

When running this script, a successful output might look like this:

(\HasChildren) "." INBOX
(\HasNoChildren) "." INBOX.Spam
(\HasNoChildren) "." INBOX.Drafts
(\HasNoChildren) "." INBOX.Sent
(\HasNoChildren) "." INBOX.Trash

If your credentials don’t work you’ll see an error message like this:

Traceback (most recent call last):
  File "./imaptest.py", line 5, in <module>
    server.login('email@domain.com', 'mypassword')
  File "/usr/lib/python3.6/imaplib.py", line 598, in login
    raise self.error(dat[-1])
imaplib.error: b'[AUTHENTICATIONFAILED] Authentication failed.'

Note that in order to be able to server.close() the connection, it’s required that you server.select() a mailbox first ; this is why we can’t just omit the server.select("INBOX") line even though we don’t actually do anything with the mailbox. See this post for a more concise example on this behaviour.

Problem:

You have IMAP code in Python similar to

server = imaplib.IMAP4_SSL('imap.mydomain.com')
server.login('email@mydomain.com', 'password')
# ...
# Cleanup
server.close()

but when you run it, server.close() fails with an error message like

Traceback (most recent call last):
  File "./imaptest.py", line 13, in <module>
    server.close()
  File "/usr/lib/python3.6/imaplib.py", line 461, in close
    typ, dat = self._simple_command('CLOSE')
  File "/usr/lib/python3.6/imaplib.py", line 1196, in _simple_command
    return self._command_complete(name, self._command(name, *args))
  File "/usr/lib/python3.6/imaplib.py", line 944, in _command
    ', '.join(Commands[name])))
imaplib.error: command CLOSE illegal in state AUTH, only allowed in states SELECTED

Solution:

Prior to server.close(), you must run server.select() at least once. If in doubt, just server.select("INBOX")because this will always work.

Insert this line before server.close():

server.select("INBOX")

It should look like this:

server = imaplib.IMAP4_SSL('imap.mydomain.com')
server.login('email@mydomain.com', 'password')
# ...
# Cleanup
server.select("INBOX")
server.close()

For a complete example see Minimal Python IMAP over SSL example

Note: Consider using IMAP with TLS instead of IMAP over SSL. See Minimal Python IMAP over TLS example.

This example code will login to the server using port 993 (IMAP over SSL), list the mailboxes and logout immediately.

#!/usr/bin/env python3
import imaplib

server = imaplib.IMAP4_SSL('imap.mydomain.com')
server.login('email@mydomain.com', 'password')
# Print list of mailboxes on server
code, mailboxes = server.list()
for mailbox in mailboxes:
    print(mailbox.decode("utf-8"))
# Select mailbox
server.select("INBOX")
# Cleanup
server.close()

Remember to replace:

• imap.mydomain.com with the domain name or IP address of your IMAP server
• email@mydomain.com by the email address you want to login with
• password by the password you want to login with

When running this script, a successful output might look like this:

(\HasChildren) "." INBOX
(\HasNoChildren) "." INBOX.Spam
(\HasNoChildren) "." INBOX.Drafts
(\HasNoChildren) "." INBOX.Sent
(\HasNoChildren) "." INBOX.Trash

If your credentials don’t work you’ll see an error message like this:

Traceback (most recent call last):
  File "./imaptest.py", line 5, in <module>
    server.login('email@domain.com', 'mypassword')
  File "/usr/lib/python3.6/imaplib.py", line 598, in login
    raise self.error(dat[-1])
imaplib.error: b'[AUTHENTICATIONFAILED] Authentication failed.'

Note that in order to be able to server.close() the connection, it’s required that you server.select() a mailbox first ; this is why we can’t just omit the server.select("INBOX") line even though we don’t actually do anything with the mailbox. See this post for a more concise example on this behaviour.