Programming languages

Problem:

You are trying to yaml.safe_dump() an object which is (or contains) a namedtuple instance such as:

import yaml
import collections

MyTuple = collections.namedtuple("MyTuple", ["a", "b", "c"])
mytuple = MyTuple(1,2,3)

yaml.safe_dump(mytuple)

which results in the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/uli/.local/lib/python3.10/site-packages/yaml/__init__.py", line 269, in safe_dump
    return dump_all([data], stream, Dumper=SafeDumper, **kwds)
  File "/home/uli/.local/lib/python3.10/site-packages/yaml/__init__.py", line 241, in dump_all
    dumper.represent(data)
  File "/home/uli/.local/lib/python3.10/site-packages/yaml/representer.py", line 27, in represent
    node = self.represent_data(data)
  File "/home/uli/.local/lib/python3.10/site-packages/yaml/representer.py", line 58, in represent_data
    node = self.yaml_representers[None](self, data)
  File "/home/uli/.local/lib/python3.10/site-packages/yaml/representer.py", line 231, in represent_undefined
    raise RepresenterError("cannot represent an object", data)
yaml.representer.RepresenterError: ('cannot represent an object', MyTuple(a=1, b=2, c=3))

Solution:

You need to add a custom representer to implicitly convert your namedtuple to a dict.

Before running yaml.safe_dump(), add the following lines:

import collections

def represent_namedtuple(dumper, data):
    return dumper.represent_dict(data._asdict())

yaml.SafeDumper.add_representer(MyTuple, represent_namedtuple)

You need to add_representer() for every namedtuple you use!

Now, the yaml.safe_dump() call should work perfectly:

yaml.safe_dump(mytuple) # Returns 'a: 1\nb: 2\nc: 3\n'

Problem:

You are trying to yaml.safe_dump() an object which is (or contains) a defaultdict, e.g.:

import yaml
import collections

yaml.safe_dump(collections.defaultdict(list))

which results in the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/uli/.local/lib/python3.10/site-packages/yaml/__init__.py", line 269, in safe_dump
    return dump_all([data], stream, Dumper=SafeDumper, **kwds)
  File "/home/uli/.local/lib/python3.10/site-packages/yaml/__init__.py", line 241, in dump_all
    dumper.represent(data)
  File "/home/uli/.local/lib/python3.10/site-packages/yaml/representer.py", line 27, in represent
    node = self.represent_data(data)
  File "/home/uli/.local/lib/python3.10/site-packages/yaml/representer.py", line 58, in represent_data
    node = self.yaml_representers[None](self, data)
  File "/home/uli/.local/lib/python3.10/site-packages/yaml/representer.py", line 231, in represent_undefined
    raise RepresenterError("cannot represent an object", data)
yaml.representer.RepresenterError: ('cannot represent an object', defaultdict(<class 'list'>, {}))

Solution:

You need to add a custom representer to implicitly convert the defaultdict to a dict.

Before running yaml.safe_dump(), add the following lines:

import collections
from yaml.representer import Representer

yaml.SafeDumper.add_representer(collections.defaultdict, Representer.represent_dict)

Now, the yaml.safe_dump() call should work perfectly:

yaml.safe_dump(collections.defaultdict(list)) # Returns'{}\n'

This simple Python script will list all files sorted by filesize in the given directory recursively. You must give an -e/--extension argument to only consider specific filename extensions. Filename extensions are compared in a case-insensitive manner.

#!/usr/bin/env python3
import os
import argparse

def convert_size(size_bytes):
    """ Convert byte size to a human-readable format. """
    for unit in ['B', 'KB', 'MB', 'GB', 'TB', 'PB', 'EB']:
        if size_bytes < 1024:
            return f"{size_bytes:.2f}{unit}"
        size_bytes /= 1024
    return f"{size_bytes:.2f}YB"

def get_files_with_extensions(root_dir, extensions):
    """ Recursively find files with the given extensions in the directory. """
    for root, dirs, files in os.walk(root_dir):
        for file in files:
            if any(file.lower().endswith(ext.lower()) for ext in extensions):
                full_path = os.path.join(root, file)
                size = os.path.getsize(full_path)
                yield full_path, size

def main():
    parser = argparse.ArgumentParser(description="List files with specific extensions sorted by size.")
    parser.add_argument("directory", type=str, help="Directory to search")
    parser.add_argument("-e", "--extension", type=str, action='append', help="File extension to filter by", required=True)
    args = parser.parse_args()

    files = list(get_files_with_extensions(args.directory, args.extension))
    files.sort(key=lambda x: x[1])

    for file, size in files:
        print(f"{convert_size(size)} - {file}")

if __name__ == "__main__":
    main()

Usage example: Find largest Python files:

python ListLargestFiles.py -e py  .

Usage example: Find (some types of) movies:

python ListLargestFiles.py -e mov -e avi -e mkv -e mp4 ~/Nextcloud

Example output:

[...]
108.81MB - /home/uli/Nextcloud/Google Fotos/20220925_151542.mp4
117.92MB - /home/uli/Nextcloud/Google Fotos/20220925_151958.mp4
237.51MB - /home/uli/Nextcloud/Google Fotos/20220911_115209.mp4
265.02MB - /home/uli/Nextcloud/Google Fotos/20220905_151716.mp4
317.36MB - /home/uli/Nextcloud/Google Fotos/20220912_124906.mp4
431.06MB - /home/uli/Nextcloud/Google Fotos/20220921_153051.mp4

Problem:

You intend to publish your package on PyPI using poetry using a command such as

poetry publish

However, you see the following error message:

Publishing mypackage (0.1.0) to PyPI
 - Uploading mypackage-0.1.0-py3-none-any.whl FAILED

HTTP Error 403: Invalid or non-existent authentication information. See https://pypi.org/help/#invalid-auth for more information. | b'<html>\n <head>\n  <title>403 Invalid or non-existent authentication information. See https://pypi.org/help/#invalid-auth for more information.\n \n <body>\n  <h1>403 Invalid or non-existent authentication information. See https://pypi.org/help/#invalid-auth for more information.\n  Access was denied to this resource.<br/><br/>\nInvalid or non-existent authentication information. See https://pypi.org/help/#invalid-auth for more information.\n\n\n \n'

Solution:

First, you need to create a PyPI API token if you don’t have one already. The username/password authentication is deprecated.

Now run the following command to configure poetry:

poetry config http-basic.pypi __token__ APITOKEN

where APITOKEN is the API token you want to use on this computer. Leave __token__ as-is since it’s a special username that tells PyPI to use an API token instead of the old & deprecated username/password authentication

It should look like this:

poetry config http-basic.pypi __token__ pypi-AgEIc[...]

Now retry publishing:

poetry publish

Example output:

Publishing mypackage (0.1.0) to PyPI
 - Uploading mypackage-0.1.0-py3-none-any.whl 100%
 - Uploading mypackage-0.1.0.tar.gz 100%

I recommend Poetry for initializing, managing & publishing python projects.