Multiple-interpreter zipapp

[gsemet]

Hello.

I have more and more dependencies on system specific libraries (ex: pydantic). So my zipapp is only for one system + 1 interpreter version : linux64 Python 3.10 for instance.

How can I create a multiple interpreter zipapp (at least for Linux a single zipapp for Python 3.9-3.12 for instance )?

[oh-ok]

I don't think shiv has a convenient way to do this, (mostly because pip any python have no way of separating dependencies by Python version at runtime).

PEX can do this out of the box by bundling a wheel for each specified platform and version combo, and then picking from them at runtime.

If you're keen on using shiv, you can try installing your dependencies to separate folders, and then merging those directories with shiv using multiple --site-packages.

This only works if the only difference between the Python 3.x and 3.y version of a package is the native binary, since the directories get merged in the zip. It also relies on pre-built wheels being available on PyPI. Here's an example using PyYAML, which has a __with_libyaml__ property to tell if you're using the native accelerator.

[test]$ cat requirements.txt
PyYAML==6.0.2
[test]$ VERS="39 310 311 312"
[test]$ for VER in $VERS; do
    pip install --only-binary=:all: --python-version=${VER} --target ./site-packages${VER} -r requirements.txt
done
...

[test]$ shiv $(printf ' --site-packages=%s' ./site-packages*) -o test.pyz
[test]$ python3.13 ./test.pyz -c 'import sys, yaml; print(sys.version_info[:2], yaml.__with_libyaml__)'
(3, 13) False
[test]$ python3.11 ./test.pyz -c 'import sys, yaml; print(sys.version_info[:2], yaml.__with_libyaml__)'
(3, 11) True
[test]$ python3.9  ./test.pyz -c 'import sys, yaml; print(sys.version_info[:2], yaml.__with_libyaml__)'
(3, 9) True

We didn't package for python 3.13, so that is False, but the ones we did package are all True.