pymethods: add support for __del__

[messense]

Adds support for __del__ in #[pymethods], mapped to CPython's tp_finalize slot.

#[pymethods]
impl MyClass {
    fn __del__(&mut self) {
        println!("instance is being destroyed");
    }
}

Closes #2479

[davidhewitt]

Thanks! Please also add a CHANGELOG and add to protocols.md in the guide.

Note that also on Python 3.7 we need to set Py_TPFLAGS_HAVE_FINALIZE, it's ignored on newer versions.

[davidhewitt]

I found today that if we don't implement tp_dealloc then the default tp_dealloc implementation (at least for CPython) will call tp_finalize. This gives us a potential route to support this and also fix #3064 at the same time.

However the documentation in this area is confusing, as it implies that tp_dealloc doesn't have a default - python/cpython#105018 / https://docs.python.org/3/c-api/typeobj.html#c.PyTypeObject.tp_dealloc

I think it may be possible that we use that default implementation to support this case, but we'll need to test carefully whether this also works for all Python versions, PyPy etc.

[davidhewitt]

I spoke to vstinner (thought no need to ping so no @) briefly about this, I understood that PyObject_CallFinalizerFromDealloc could be added to the limited API in 3.13. So maybe we can figure out a way to proceed with this where __del__ is supported without abi3 for now and then we can add abi3 support on 3.13.

[codspeed-hq]

Merging this PR will not alter performance

✅ 105 untouched benchmarks
⏩ 1 skipped benchmark¹

---

_{Comparing messense:del (322cef6) with main (bd04fcd)}

Footnotes

1. 1 benchmark was skipped, so the baseline result was used instead. If it was deleted from the codebase, click here and archive it to remove it from the performance reports. ↩

[messense]

With the help from LLM, made some progress on this, tests are passing, review would be appreciated!

I have also looked into removing our tp_dealloc implementation but turns out to be pretty hard to get everything right.

[davidhewitt]

Thanks for moving this forward! There are a few typical LLM warts like duplicate tests, I think we might also be able to simplify a bit.

I'd like the behavior for abi3 to be well defined for users. The two options I see:

• Don't allow __del__ on abi3 for now.
• Allow __del__ on abi3 with Python 3.9 (see my comments with ideas how to support this below)

I also opened python/cpython#146063 to ask upstream to add the needed methods to the stable ABI (probably I should have done this years ago).

[davidhewitt]

I have also looked into removing our tp_dealloc implementation but turns out to be pretty hard to get everything right.

I think it's probably necessary for us to continue to implement tp_dealloc, I don't think there's an alternative way for us to drop the Rust data within types at the point of destruction.