Python

This will download page 1000 … page 1 in descending order and save the HTML to 1000.html up to 1.html

import requests
for i in range(1000, 0, -1):
    with open(f"{i}.html", "w") as outfile:
        outfile.write(requests.get(f"https://forum.my-domain.com/showthread.php?123456-my-thread/page{i}").text)

With progress bar

import requests
from tqdm import tqdm
for i in tqdm(range(1000, 0, -1)):
    with open(f"{i}.html", "w") as outfile:
        outfile.write(requests.get(f"https://forum.my-domain.com/showthread.php?123456-my-thread/page{i}").text)

Also see How to find first element in sorted numpy array that is smaller than a given scalar

You can use np.searchsorted() like this:

idx = np.searchsorted(arr, scalar, side='right') + 1

Full example:

import numpy as np

# Example array
arr = np.array([1, 2, 3, 4, 5])

# We'll search for this value
scalar = 3.5

# Use numpy.searchsorted() to find the first element in the array that is larger than the scalar
# We need to use + 1 since searchsorted() returns the first element that is smaller than the scalar
idx = np.searchsorted(arr, scalar, side='right') + 1

# Print the resulting index
print(idx)

Also see How to find first element in sorted numpy array that is larger than a given scalar

You can use np.searchsorted() like this:

idx = np.searchsorted(arr, scalar, side='left')

Full example:

import numpy as np

# Example array
arr = np.array([1, 2, 3, 4, 5])

# We'll search for this value
scalar = 3.5

# Use numpy.searchsorted() to find the first element in the array that is smaller than the scalar
idx = np.searchsorted(arr, scalar, side='left')

# Print the resulting index
print(idx)

You can use the tqdm library to easily add a progress bar to your Python for i in range(...) loop:

Just

from tqdm import tqdm

at the top of your script and then surround range(...) with tqdm(...). For example,

mysum = 0
for i in range(50):
    sum += i

becomes

mysum = 0
for i in tqdm(range(50)):
    mysum += i

Want to have the same size as a but a different data type? See How to create NumPy array that is exactly as long as another array but has a different data type

If you have a numpy array such as

a = np.arange(50)

you can use the following code to create an array b that has the same dimensions, i.e. the same size, and the same datataype as a using

b = np.zeros_like(a)

Want to have the same data type as a as well as the same size? See How to create NumPy array that is exactly as long as another array

If you have a numpy array such as

a = np.arange(50) * 1.1

you can use the following code to create an array b that has the same dimensions, i.e. the same size as a, but uses the np.intinteger data type as opposed to the floating point data type of a:

b = np.zeros_like(a, dtype=np.int)

Use .ndim == 1 which contains the number of dimensions in the array.

if a.ndim == 1:
    print('a is a 1D array')
else:
    print('a is not a 1D array')

You can also use assert to check that:

assert a.ndim == 1

which will raise an AssertionError if a is not a one-dimensional array.

def is_platform_windows()
    return platform.system() == "Windows"

def is_platform_linux():
    return platform.system() == "Linux"

Problem:

While trying to write an object to a file using msgpack using code like

with open("myobj.msgpack", "w") as outfile:
    msgpack.dump(myobj, outfile)

you see an error message like

File /usr/lib/python3/dist-packages/msgpack/__init__.py:26, in pack(o, stream, **kwargs)
     20 """
     21 Pack object `o` and write it to `stream`
     22 
     23 See :class:`Packer` for options.
     24 """
     25 packer = Packer(**kwargs)
---> 26 stream.write(packer.pack(o))

TypeError: write() argument must be str, not bytes

Solution:

msgpack writes binary data, so you need to open the file in binary mode using open(..., "wb"):

with open("myobj.msgpack", "wb") as outfile:
    msgpack.dump(myobj, outfile)

In our previous post How to reload import in Jupyter we showed how to use reload() from importlib in order to reload a module without restarting the kernel.

import mymodule
# Reload .py file every time we run the cell
from importlib import reload
reload(mymodule)

However, often we are using syntax like

from mymodule import *

to wildcard-load everything imported from mymodule.

We can reload those wildcard imports using the same strategy by additionalling importing mymodule and then reloading using reload(mymodule) – this will also reload the wildcard import from mymodule import *!

from mymodule import *

# This line is just to facilitate reloading
import mymodule
from importlib import reload
reload(mymodule)
# Reload .py file every time we run the cell from importlib import reload reload(mymodule)

If you have a date like

import arrow

dt = arrow.get("2021-05-27")

you can add N months (example: 18 months) to said date using

dt.shift(months=+18)

The result will be an arrow representation of 2022-11-27