Programming languages

This function sums up all values from the given 3D array:

def primitive_pixel_sum(frame):
    result = 0.0
    for x in range(frame.shape[0]):
        for y in range(frame.shape[1]):
            for z in range(frame.shape[2]):
                result += frame[x,y,z]
    return result

Whereas the following function is exactly the same algorithm, but using numba.jit:

import numba

@numba.jit
def numba_pixel_sum(frame):
    result = 0.0
    for x in range(frame.shape[0]):
        for y in range(frame.shape[1]):
            for z in range(frame.shape[2]):
                result += frame[x,y,z]
    return result

We can benchmark them in Jupyter using

%%timeit
primitive_pixel_sum(frame)

and

%%timeit
numba_pixel_sum(frame)

respectively.

Results

We tested this with a random camera image taken from OpenCV of shape (480, 640, 3)

primitive_pixel_sum():

1.78 s ± 253 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

numba_pixel_sum():

4.06 ms ± 413 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)

It should be clear from these results that the numba version is 438 times faster compared to the primitive version.

Note that when compiling complex functions using numba.jit it can take many milliseconds or even seconds to compile – possibly longer than a simple Python function would take.

Since it’s so simple to use Numba, my recommendation is to just try it out for every function you suspect will eat up a lot of CPU time. Over time you will be able to develop an intuition for which functions it’s worth to use Numba and which functions won’t work at all or if it will be slower overall than just using Python.

Remember than often you can also use NumPy functions to achieve the same result. In our example, you could achieve the same thing using

np.sum(frame)

which is even faster than Numba:

%%timeit
np.sum(frame)

Result:

2.5 ms ± 7.17 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In bottle you can generate a custom HTTP response code by first importing HTTPResponse:

from bottle import HTTPResponse

and then, in your @route function, raise a HTTPResponse, for example

raise HTTPResponse("This is a HTTP 401 test URL", status=401)

Full example

#!/usr/bin/env python3
from bottle import HTTPResponse, route, run

@route('/test401')
def hello():
    raise HTTPResponse("This is a HTTP 401 test URL", status=401)

run(host='localhost', port=8080, debug=True)

Run this, script, then open http://localhost/test401 which will result in a HTTP 401 (unauthorized) response code with

This is a HTTP 401 test URL

as response body.

In order to obtain the value of a query parameter named myparam in bottle, first import request:

from bottle import request

and then use

request.params["myparam"]

Full example:

#!/usr/bin/env python3
from bottle import route, run, request

@route('/test')
def hello():
    query_param = request.params["myparam"]
    return f"myparam is {query_param}"

run(host='localhost', port=8080, debug=True)

Run this script and open http://localhost:8080/test?myparam=xyz in your browser – you will see

myparam is xyz

This script copies myfolder to myfolder2 recursively and renames all .txt files to .xml

#!/usr/bin/env python3
import os
import os.path
import re
import shutil

srcfolder = "myfolder"
dstfolder = "myfolder2"

for subdir, dirs, files in os.walk(srcfolder):
    for file in files:
        # Rename .txt to .xml
        if file.endswith(".txt"):
            dstfile_name = re.sub(r"\.txt$", ".xml", file) # Regex to replace .txt only at the end of the string
        else:
            dstfile_name = file
        # Compute destination path
        srcpath = os.path.join(subdir, file)
        dstpath_relative_to_outdir = os.path.relpath(os.path.join(subdir, dstfile_name), start=srcfolder)
        dstpath = os.path.join(dstfolder, dstpath_relative_to_outdir)
        # Create directory if not exists
        os.makedirs(os.path.dirname(dstpath), exist_ok=True)
        # Copy file (copy2 preserves metadata)
        shutil.copy2(srcpath, dstpath)

This script will zip the folder myfolder recursively to myzip. Note that empty directories will not be copied over to the ZIP.

#!/usr/bin/env python3
import zipfile
import os

# This folder 
folder = "myfolder"

with zipfile.ZipFile(f"{folder}.zip", "w") as outzip:
    for subdir, dirs, files in os.walk(folder):
        for file in files:
            # Read file
            srcpath = os.path.join(subdir, file)
            dstpath_in_zip = os.path.relpath(srcpath, start=folder)
            with open(srcpath, 'rb') as infile:
                # Write to zip
                outzip.writestr(dstpath_in_zip, infile.read())