How to fix Python Pillow pip install exception: RequiredDependencyException: jpeg

Problem:

When trying to install pillow such as using

pip install Pillow

you see an error log like

      running build_ext
      
      
      The headers or library files could not be found for jpeg,
      a required dependency when compiling Pillow from source.
      
      Please see the install instructions at:
         https://pillow.readthedocs.io/en/latest/installation.html
      
      Traceback (most recent call last):
        File "/tmp/pip-install-_g5fa7ox/pillow_7cb18c0d6bec468e8844184b98c8bf45/setup.py", line 989, in <module>
          setup(
        File "/home/uli/.pypy3-virtualenv/lib/pypy3.9/site-packages/setuptools/__init__.py", line 87, in setup
          return distutils.core.setup(**attrs)
        File "/home/uli/.pypy3-virtualenv/lib/pypy3.9/site-packages/setuptools/_distutils/core.py", line 148, in setup
          return run_commands(dist)
        File "/home/uli/.pypy3-virtualenv/lib/pypy3.9/site-packages/setuptools/_distutils/core.py", line 163, in run_commands
          dist.run_commands()
        File "/home/uli/.pypy3-virtualenv/lib/pypy3.9/site-packages/setuptools/_distutils/dist.py", line 967, in run_commands
          self.run_command(cmd)
        File "/home/uli/.pypy3-virtualenv/lib/pypy3.9/site-packages/setuptools/dist.py", line 1214, in run_command
          super().run_command(command)
        File "/home/uli/.pypy3-virtualenv/lib/pypy3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
          cmd_obj.run()
        File "/home/uli/.pypy3-virtualenv/lib/pypy3.9/site-packages/setuptools/command/install.py", line 68, in run
          return orig.install.run(self)
        File "/home/uli/.pypy3-virtualenv/lib/pypy3.9/site-packages/setuptools/_distutils/command/install.py", line 670, in run
          self.run_command('build')
        File "/home/uli/.pypy3-virtualenv/lib/pypy3.9/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/home/uli/.pypy3-virtualenv/lib/pypy3.9/site-packages/setuptools/dist.py", line 1214, in run_command
          super().run_command(command)
        File "/home/uli/.pypy3-virtualenv/lib/pypy3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
          cmd_obj.run()
        File "/home/uli/.pypy3-virtualenv/lib/pypy3.9/site-packages/setuptools/_distutils/command/build.py", line 136, in run
          self.run_command(cmd_name)
        File "/home/uli/.pypy3-virtualenv/lib/pypy3.9/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/home/uli/.pypy3-virtualenv/lib/pypy3.9/site-packages/setuptools/dist.py", line 1214, in run_command
          super().run_command(command)
        File "/home/uli/.pypy3-virtualenv/lib/pypy3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
          cmd_obj.run()
        File "/home/uli/.pypy3-virtualenv/lib/pypy3.9/site-packages/setuptools/command/build_ext.py", line 79, in run
          _build_ext.run(self)
        File "/home/uli/.pypy3-virtualenv/lib/pypy3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 339, in run
          self.build_extensions()
        File "/tmp/pip-install-_g5fa7ox/pillow_7cb18c0d6bec468e8844184b98c8bf45/setup.py", line 804, in build_extensions
          raise RequiredDependencyException(f)
      RequiredDependencyException: jpeg
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-_g5fa7ox/pillow_7cb18c0d6bec468e8844184b98c8bf45/setup.py", line 1009, in <module>
          raise RequiredDependencyException(msg)
      RequiredDependencyException:
      
      The headers or library files could not be found for jpeg,
      a required dependency when compiling Pillow from source.
      
      Please see the install instructions at:
         https://pillow.readthedocs.io/en/latest/installation.html
      
      
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> Pillow

Solution:

Pillow needs a buch of libraries to be installed in order to work properly. Use the following command from the official Pillow website on Ubuntu:

sudo apt-get install cmake libtiff5-dev libjpeg8-dev libopenjp2-7-dev zlib1g-dev libfreetype6-dev liblcms2-dev libwebp-dev tcl8.6-dev tk8.6-dev python3-tk libharfbuzz-dev libfribidi-dev libxcb1-dev

or check out the installation guide for commands for other operating systems.

Posted by Uli Köhler in Python

How to install tailscale on Fedora CoreOS

In order to install tailscale, on Fedora CoreOS (this post has been tested on Fedora CoreOS 35), you can use this sequence of commands:

sudo curl -o /etc/yum.repos.d/tailscale.repo https://pkgs.tailscale.com/stable/fedora/tailscale.repo
sudo rpm-ostree install tailscale

Now reboot using

sudo systemctl reboot

Once rebooted, you can enable the service using

sudo systemctl enable --now tailscaled

and then configure tailscale as usual:

sudo tailscale up --login-server .... --authkey ...

Also see our post on How to connect tailscale to headscale server on Linux

Posted by Uli Köhler in CoreOS, Headscale, VPN

How to fix Python MongoDB TypeError: Object of type ObjectId is not JSON serializable

Problem:

When trying to export data as JSON that has originally been queried from MongoDB using code like

with open("alle.json", "w") as outfile:
    json.dump(alle, outfile)

you see the following error message:

File /usr/lib/python3.9/json/__init__.py:179, in dump(obj, fp, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, default, sort_keys, **kw)
    173     iterable = cls(skipkeys=skipkeys, ensure_ascii=ensure_ascii,
    174         check_circular=check_circular, allow_nan=allow_nan, indent=indent,
    175         separators=separators,
    176         default=default, sort_keys=sort_keys, **kw).iterencode(obj)
    177 # could accelerate with writelines in some versions of Python, at
    178 # a debuggability cost
--> 179 for chunk in iterable:
    180     fp.write(chunk)

File /usr/lib/python3.9/json/encoder.py:429, in _make_iterencode.<locals>._iterencode(o, _current_indent_level)
    427     yield _floatstr(o)
    428 elif isinstance(o, (list, tuple)):
--> 429     yield from _iterencode_list(o, _current_indent_level)
    430 elif isinstance(o, dict):
    431     yield from _iterencode_dict(o, _current_indent_level)

File /usr/lib/python3.9/json/encoder.py:325, in _make_iterencode.<locals>._iterencode_list(lst, _current_indent_level)
    323         else:
    324             chunks = _iterencode(value, _current_indent_level)
--> 325         yield from chunks
    326 if newline_indent is not None:
    327     _current_indent_level -= 1

File /usr/lib/python3.9/json/encoder.py:405, in _make_iterencode.<locals>._iterencode_dict(dct, _current_indent_level)
    403         else:
    404             chunks = _iterencode(value, _current_indent_level)
--> 405         yield from chunks
    406 if newline_indent is not None:
    407     _current_indent_level -= 1

File /usr/lib/python3.9/json/encoder.py:438, in _make_iterencode.<locals>._iterencode(o, _current_indent_level)
    436         raise ValueError("Circular reference detected")
    437     markers[markerid] = o
--> 438 o = _default(o)
    439 yield from _iterencode(o, _current_indent_level)
    440 if markers is not None:

File /usr/lib/python3.9/json/encoder.py:179, in JSONEncoder.default(self, o)
    160 def default(self, o):
    161     """Implement this method in a subclass such that it returns
    162     a serializable object for ``o``, or calls the base implementation
    163     (to raise a ``TypeError``).
   (...)
    177 
    178     """
--> 179     raise TypeError(f'Object of type {o.__class__.__name__} '
    180                     f'is not JSON serializable')

TypeError: Object of type ObjectId is not JSON serializable

Solution:

This error occurs because objects queried from PyMongo always contain _id which is of type ObjectId and the normal JSON library (or drop-in replacements like simplejson do not know how to create JSON representations of Objects of type ObjectId).

In order to fix this, use pymongo‘s json_util instead of json. Note that the bson.json_util package contains dumps but does not contain dump, so use the following snippet to write to a file:

 

import bson.json_util as json_util

with open("alle.json", "w") as outfile:
    outfile.write(json_util.dumps(alle))

 

Posted by Uli Köhler in MongoDB, Python

How to fix RcppGSL installation error gsl-config: Command not found

Problem:

While trying to install RcppGSL using

BiocManager::install("RcppGSL")

you see the following error message:

checking for gcc option to accept ISO C89... none needed
checking for gsl-config... no
configure: error: gsl-config not found, is GSL installed?
ERROR: configuration failed for package ‘RcppGSL’
* removing ‘/usr/local/lib/R/site-library/RcppGSL’

The downloaded source packages are in
        ‘/tmp/RtmpqSzFab/downloaded_packages’
Warning message:
In .inet_warning(msg) :
  installation of package ‘RcppGSL’ had non-zero exit status

Solution:

You need to install the libgsl2 development headers which include the gsl-config executable.

On Ubuntu, you can install it using

sudo apt -y install libgsl2-dev

 

Posted by Uli Köhler in R

How to iterate all databases in PyMongo

This short example shows how to iterate all databases or list all database names for a MongoDB in Python using pymongo:

from pymongo import MongoClient

client = MongoClient("mongodb://localhost")

for database in client.list_databases():
    print(database['name'])

s

Posted by Uli Köhler in Python

How to start Jupyter Lab for remote access

This will start Jupyter listening on all network interfaces / bind to all IP addresses in order to make direct browser access possible not only from localhost but any remote host that has network access to the host where you’re running Jupyter:

jupyter lab --ip=0.0.0.0

 

Posted by Uli Köhler in Networking, Python

How to install gcsfuse on Ubuntu in 15 seconds

export GCSFUSE_REPO=gcsfuse-`lsb_release -c -s`
echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" | sudo tee /etc/apt/sources.list.d/gcsfuse.list
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
sudo apt-get update
sudo apt-get install gcsfuse

If that doesn’t work (such as for Ubuntu 21.10 impish at the time of writing this post), use the following method:

curl -L -O https://github.com/GoogleCloudPlatform/gcsfuse/releases/download/v0.39.2/gcsfuse_0.39.2_amd64.deb
sudo dpkg --install gcsfuse_0.39.2_amd64.deb
rm gcsfuse_0.39.2_amd64.deb

 

 

This is a summary from the official docs.

Posted by Uli Köhler in Cloud, Linux

How to fix gsutil 401 Anonymous caller does not have storage.objects.list access to the Google Cloud Storage bucket

Problem:

While running a command like

gsutil rsync my-folder gs://my-bucket

you see an error message like

Building synchronization state...
Caught non-retryable exception while listing gs://mfwh-backups/: ServiceException: 401 Anonymous caller does not have storage.objects.list access to the Google Cloud Storage bucket.
CommandException: Caught non-retryable exception - aborting rsync

Solution:

This error is basically telling you that you are not logged in !

First, create a service account for the project on Google cloud: Direct link to service account page. You need to figure out depending on your setup what roles you want to assign to the service account. If you are lost and don’t know what to select, just assign it admin rights on the storage, but be aware that this might have security implications, as this account may also delete or create storage buckets etc.

Then open the page for that service account and create a new key!

This will give you a JSON file such as my-project-4d267a915c4e.json. Save it on the server or computer where you want to run gsutil. I recommend to save it in ~ (the user’s home folder) with the original filename, for example ~/my-project-4d267a915c4e.json.

Then you need to activate that service account using

gcloud auth activate-service-account --key-file [path to JSON file]

such as

gcloud auth activate-service-account --key-file [path to JSON file]

 

Posted by Uli Köhler in Cloud

How to install gcloud on Ubuntu in 10 seconds

sudo snap install google-cloud-cli --classic

This is the summary from the official docs. I recommend to install it as snap package as opposed to a deb package since it will auto-update, it’s much easier to use and just works better out of the box in my experience.

Posted by Uli Köhler in Cloud, Linux

How to fix R package installation fatal error: lzma.h: No such file or directory

Problem:

While installing some R package, you see an error message like

cram/cram_io.c:61:10: fatal error: lzma.h: No such file or directory
   61 | #include 
      |          ^~~~~~~~
compilation terminated.

Solution:

You need to install the liblzma development headers. On Ubuntu, you can do that using

sudo apt -y install liblzma-dev

 

Posted by Uli Köhler in R

How to fix R package installation fatal error: bzlib.h: No such file or directory

Problem:

While installing some R package, you see an error message like

cram/cram_io.c:57:10: fatal error: bzlib.h: No such file or directory
   57 | #include 
      |          ^~~~~~~~~
compilation terminated.

Solution:

You need to install the libbz2 development headers. On Ubuntu, you can do that using

sudo apt -y install libbz2-dev

 

Posted by Uli Köhler in R

How to fix Jupyter Lab ImportError: cannot import name ‘soft_unicode’ from ‘markupsafe’

Problem:

When running

jupyter lab

you see the following error message:

Traceback (most recent call last):
  File "/usr/local/bin/jupyter-lab", line 5, in <module>
    from jupyterlab.labapp import main
  File "/usr/local/lib/python3.8/dist-packages/jupyterlab/labapp.py", line 13, in <module>
    from jupyter_server.serverapp import flags
  File "/usr/local/lib/python3.8/dist-packages/jupyter_server/serverapp.py", line 39, in <module>
    from jinja2 import Environment, FileSystemLoader
  File "/usr/lib/python3/dist-packages/jinja2/__init__.py", line 33, in <module>
    from jinja2.environment import Environment, Template
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 15, in <module>
    from jinja2 import nodes
  File "/usr/lib/python3/dist-packages/jinja2/nodes.py", line 23, in <module>
    from jinja2.utils import Markup
  File "/usr/lib/python3/dist-packages/jinja2/utils.py", line 656, in <module>
    from markupsafe import Markup, escape, soft_unicode
ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.8/dist-packages/markupsafe/__init__.py)

Solution:

You need to install an older version of markupsafe using

sudo pip3 install markupsafe==2.0.1

until other packages have been updated.

Posted by Uli Köhler in Python

How to fix R package installation /usr/bin/ld: cannot find -lblas

Problem:

While installing some R package, you see an error message like

/usr/bin/ld: cannot find -lblas

Solution:

You need to install the libblas development headers. On Ubuntu, you can do that using

sudo apt -y install libblas-dev

 

Posted by Uli Köhler in R

How to fix R package installation /usr/bin/ld: cannot find -llapack

Problem:

While installing some R package, you see an error message like

/usr/bin/ld: cannot find -llapack

Solution:

You need to install the liblapack development headers. On Ubuntu, you can do that using

sudo apt -y install liblapack-dev

 

Posted by Uli Köhler in R

How to fix R package installation fatal error: png.h: No such file or directory

Problem:

While installing some R package, you see an error message like

/bin/bash: libpng-config: command not found
read.c:3:10: fatal error: png.h: No such file or directory
    3 | #include 
      |          ^~~~~~~
compilation terminated.
make: *** [/usr/lib/R/etc/Makeconf:168: read.o] Error 1

Solution:

You need to install the libpng development headers. On Ubuntu, you can do that using

sudo apt -y install libpng-dev

 

Posted by Uli Köhler in R

How to fix R package installation fatal error: zlib.h: No such file or directory

Problem:

While installing some R package, you see an error message like

io_utils.c:16:10: fatal error: zlib.h: No such file or directory
   16 | #include <zlib.h>
      |          ^~~~~~~~
compilation terminated.
make: *** [/usr/lib/R/etc/Makeconf:168: io_utils.o] Error 1

Solution:

You need to install the zlib development headers. On Ubuntu, you can do that using

sudo apt -y install zlib1g-dev

 

Posted by Uli Köhler in R
This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish. Cookie settingsACCEPTPrivacy &amp; Cookies Policy