How to fix Python CFFI package not finding include file when running pip install

Problem:

When you install your package locally using python setup.py install, all include files are found and the package installs correctly

However you run pip install, the compiler is missing some include files

Solution

Most likely, your include file does not end up in the sdist i.e. the source distribution .tar.gz file. In order to include it, create a MANIFEST.in file in the same folder as setup.py in order to tell setuptools what to include in addition to any file directly referenced in setup.py.

In my case, the include files are in the src folder so I add this to MANIFEST.in:

graft src/

Now, to check if that worked, run

python setup.py sdist

which will show you which files are included in the source distribution. Example:

C:\Users\ukoeh\cv_algorithms>python setup.py sdist
running sdist
running egg_info
writing cv_algorithms.egg-info\PKG-INFO
writing dependency_links to cv_algorithms.egg-info\dependency_links.txt
writing requirements to cv_algorithms.egg-info\requires.txt
writing top-level names to cv_algorithms.egg-info\top_level.txt
reading manifest file 'cv_algorithms.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
writing manifest file 'cv_algorithms.egg-info\SOURCES.txt'
running check
warning: check: missing meta-data: either (author and author_email) or (maintainer and maintainer_email) should be supplied

creating cv_algorithms-1.0.4
creating cv_algorithms-1.0.4\cv_algorithms
creating cv_algorithms-1.0.4\cv_algorithms.egg-info
creating cv_algorithms-1.0.4\src
creating cv_algorithms-1.0.4\test
copying files to cv_algorithms-1.0.4...
copying LICENSE -> cv_algorithms-1.0.4
copying MANIFEST.in -> cv_algorithms-1.0.4
copying README.md -> cv_algorithms-1.0.4
copying setup.py -> cv_algorithms-1.0.4
copying cv_algorithms__init__.py -> cv_algorithms-1.0.4\cv_algorithms
copying cv_algorithms_checks.py -> cv_algorithms-1.0.4\cv_algorithms
copying cv_algorithms_ffi.py -> cv_algorithms-1.0.4\cv_algorithms
copying cv_algorithms\classification.py -> cv_algorithms-1.0.4\cv_algorithms
copying cv_algorithms\colorspace.py -> cv_algorithms-1.0.4\cv_algorithms
copying cv_algorithms\contours.py -> cv_algorithms-1.0.4\cv_algorithms
copying cv_algorithms\distance.py -> cv_algorithms-1.0.4\cv_algorithms
copying cv_algorithms\grassfire.py -> cv_algorithms-1.0.4\cv_algorithms
copying cv_algorithms\morphology.py -> cv_algorithms-1.0.4\cv_algorithms
copying cv_algorithms\neighbours.py -> cv_algorithms-1.0.4\cv_algorithms
copying cv_algorithms\popcount.py -> cv_algorithms-1.0.4\cv_algorithms
copying cv_algorithms\text.py -> cv_algorithms-1.0.4\cv_algorithms
copying cv_algorithms\thinning.py -> cv_algorithms-1.0.4\cv_algorithms
copying cv_algorithms\utils.py -> cv_algorithms-1.0.4\cv_algorithms
copying cv_algorithms.egg-info\PKG-INFO -> cv_algorithms-1.0.4\cv_algorithms.egg-info
copying cv_algorithms.egg-info\SOURCES.txt -> cv_algorithms-1.0.4\cv_algorithms.egg-info
copying cv_algorithms.egg-info\dependency_links.txt -> cv_algorithms-1.0.4\cv_algorithms.egg-info
copying cv_algorithms.egg-info\not-zip-safe -> cv_algorithms-1.0.4\cv_algorithms.egg-info
copying cv_algorithms.egg-info\requires.txt -> cv_algorithms-1.0.4\cv_algorithms.egg-info
copying cv_algorithms.egg-info\top_level.txt -> cv_algorithms-1.0.4\cv_algorithms.egg-info
copying src\common.hpp -> cv_algorithms-1.0.4\src
copying src\distance.cpp -> cv_algorithms-1.0.4\src
copying src\grassfire.cpp -> cv_algorithms-1.0.4\src
copying src\neighbours.cpp -> cv_algorithms-1.0.4\src
copying src\popcount.cpp -> cv_algorithms-1.0.4\src
copying src\thinning.cpp -> cv_algorithms-1.0.4\src
copying src\windows.cpp -> cv_algorithms-1.0.4\src
copying test\TestColorspace.py -> cv_algorithms-1.0.4\test
copying test\TestContours.py -> cv_algorithms-1.0.4\test
copying test\TestDistance.py -> cv_algorithms-1.0.4\test
copying test\TestGrassfire.py -> cv_algorithms-1.0.4\test
copying test\TestNeighbours.py -> cv_algorithms-1.0.4\test
copying test\TestPopcount.py -> cv_algorithms-1.0.4\test
copying test\TestThinning.py -> cv_algorithms-1.0.4\test
copying test\TestUtils.py -> cv_algorithms-1.0.4\test
copying test__init__.py -> cv_algorithms-1.0.4\test
Writing cv_algorithms-1.0.4\setup.cfg
Creating tar archive
removing 'cv_algorithms-1.0.4' (and everything under i

After that, you need to re-release your package. See my cv_algorithms project for a complete example.