Update editable install support for PEP 829 (`.start` files)

[rgommers]

PEP 829 – Package Startup Configuration Files is deprecating import statements in .pth files, and replaces them with entrypoints in .pth files and .start files for Python 3.15. It looks like editable installs were considered in the discussion and the replacement should be a small diff. However, no one tested that it looks like, so it's hard to be sure - we should try this soon to see if there are any unexpected issues. The implementation landed two days ago in python/cpython#149109, so should be available in 3.15.0b1

[dnicolodi]

I read PEP 829 very quickly and I am not very happy with it.

I don't understand the problem it solves: if a package can write a rogue .pth file it can also write a rogue .start file, or any other file, including a .start file that appends a module search path containing a module shadowing a standard library module, for example...

I don't see why the support for .pth files needs to be removed in three years from the release of Python 3.15: that makes wheels that would otherwise be Python version independent suddenly become Python version dependent and the wheel format does not have an easy way to say that a wheel is compatible only with Python versions <= 3.18.

I'm sure there are other oddities I didn't immediately spot.

[rgommers]

I agree it's not a super exciting PEP: it's a minor improvement, while adding churn. I did read through the discussion thread, and other people pointed out the same - it doesn't materially change the security aspects. The counterpoint seems to be that it is a step in the right direction, because taking the import (/exec()) part out of .pth files, it becomes easier to statically inspect wheels and conclude only based on filename extensions whether there's executable code.

It seems like a done deal now (it wasn't posted under packaging, so I only heard about the PEP today) - but as long as the change for us is small and it works, it's also not a big issue. If it's not easy to adopt to though, we can still push back if we're early enough.

[dnicolodi]

it becomes easier to statically inspect wheels and conclude only based on filename extensions whether there's executable code

The point of wheels is to distribute executable code, I think you mean code executed at interpreter initialization time 🙂

[rgommers]

Yes, indeed;)

[eli-schwartz]

Replacing these lines with the narrower invocation of entry point function inside modules indirectly reduces the attack vector because functions inside modules are easier to audit, both by humans and automatic vulnerability scanners.

Essentially the "benefit" of the PEP is that code which only scans .py files for suspicious code, doesn't need to be modified to also scan .pth files.

It also claims that a benefit is "en passant as part of rewriting the implementation" the new implementation uses importlib instead of exec, which apparently means that you can write your own third-party software to hook into the audit framework and observe imports; it is not clear why this needed a new file format.

[eli-schwartz]

One significant practical issue is that even if you buy into the change,

I don't see why the support for .pth files needs to be removed in three years from the release of Python 3.15: that makes wheels that would otherwise be Python version independent suddenly become Python version dependent and the wheel format does not have an easy way to say that a wheel is compatible only with Python versions <= 3.18.

... the migration path is actually (apparently) to ship both pth files and start files and then get interpreter warnings at startup in python 3.20 if your support cycle is longer than upstream cpython.

So we're back to python version dependent wheels, but at least they silently work on 3.18 (which only starts to silently ignore pth files, on the assumption that you added both pth and start, and it's gonna use the latter).