gh-149044: Implement PEP 820 – PySlot: Unified slot system for the C API#149055
gh-149044: Implement PEP 820 – PySlot: Unified slot system for the C API#149055encukou merged 54 commits intopython:mainfrom
Conversation
ZeroIntensity
left a comment
There was a problem hiding this comment.
Documentation looks pretty good. I haven't looked at all of the C code yet.
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Documentation build overview
65 files changed ·
|
ngoldbaum
left a comment
There was a problem hiding this comment.
I wrote FFI bindings for this PR for PyO3 this afternoon and spotted several minor issues.
| #define PySlot_PTR(NAME, VALUE) \ | ||
| {(NAME), PySlot_INTPTR, {0}, {(void*)(VALUE)}} | ||
|
|
||
| #define PySlot_PTR_STATIC(NAME, VALUE) \ | ||
| {(NAME), PySlot_INTPTR | PySlot_STATIC, {0}, {(void*)(VALUE)}} |
There was a problem hiding this comment.
Can you spell these last two with the explicit names of the struct fields like the ones above? Also while you're touching this code, IMO these two definitions should come before PySlot_END.
There was a problem hiding this comment.
These macros are for code compatible with C++11 and below, which doesn't support designated initializers (that is, the explicit names).
So, both style issues are intentional :) I added a comment.
| #define PySlot_STATIC 0x02 | ||
| #define PySlot_INTPTR 0x04 | ||
|
|
||
| #define Py_slot_invalid 0xffff |
There was a problem hiding this comment.
for consistency should this be called Py_slot_INVALID maybe?
There was a problem hiding this comment.
That would be consistent with the wrong thing. Slot IDs are named Py_tp_repr, Py_mod_exec, etc. Flags are a new thing, so they follow the current style guide: mixed prefix + uppercase.
| #define _Py_SLOT_COMPAT_VALUE(OLD, NEW) OLD | ||
| #endif | ||
|
|
||
| #define Py_slot_end 0 |
There was a problem hiding this comment.
Maybe worth adding a comment above these that they're all u16 values?
There was a problem hiding this comment.
In C, the preprocessor macros are just numbers :)
In PyType_Slot, these values are used as int.
| #define Py_tp_finalize 80 | ||
| #define Py_am_send 81 | ||
| #define Py_tp_vectorcall 82 | ||
| #define Py_tp_token 83 |
There was a problem hiding this comment.
Everything through here is duplicated with typeslots.h. Was that intentional? It probably makes sense for there to be only one source of truth for these constants.
There was a problem hiding this comment.
Thanks for the catch! I seem to have dropped the commit that removes the unused file.
| #ifndef _Py_HAVE_SLOTS_H | ||
| #define _Py_HAVE_SLOTS_H | ||
|
|
||
| typedef void (*_Py_funcptr_t)(void); |
There was a problem hiding this comment.
PyO3 needs to wrap this type directly so you might as well give this a public non-underscored name.
There was a problem hiding this comment.
Can PyO3 use the underlying type, or better yet, some “generic C function pointer” type?
There was a problem hiding this comment.
Unfortunately no, because Rust doesn't let you cast between function pointer types without using unsafe and std::mem::transmute. That said, in my second draft of this, I isolated the unsafety into a PySlot_FUNC rust macro that hides the wrapped function pointer, so I take back my comment from last night.
There was a problem hiding this comment.
Keep in mind this is not the correct function type, just a function type. At this level, unsafe transmuation is exactly what you need in Rust.
(A safer layer on top of the ABI would need to have separate C-specific and C++-specific variants, and presumably a Rust-specific one as well. I didn't get that into 3.15.)
|
I intend to merge this today (or early tomorrow) to get it in 3.15, as postponing to 3.16 would be very inconvenient. Beta is for fixing features (or removing, if this turns out to be a bad idea after all). |
ZeroIntensity
left a comment
There was a problem hiding this comment.
I think this is in a mergeable state. I read through all of the PEP, documentation, and code, and I couldn't find any glaring issues.
I left a number of non-blocking questions/comments; feel free to ignore them for now and address them after the beta freeze.
| self.assertRaisesRegex( | ||
| SystemError, | ||
| "Py_tp_bases is not a tuple", | ||
| TypeError, | ||
| "metaclass conflict", | ||
| _testcapi.create_heapctype_with_none_bases_slot | ||
| ) |
There was a problem hiding this comment.
Hmm, this seems like a downgrade in terms of the error message. I don't see a change to create_heapctype_with_none_bases_slot in this PR, so is there a reason the error changed?
There was a problem hiding this comment.
Yeah, that's a side effect from merging Py_tp_base & Py_tp_bases -- the latter strictly required a tuple of types, so it had this message.
This is a silly restriction. I'll document lifting it.
There was a problem hiding this comment.
(And I'll get SC permission to update the PEP, or revert this.)
| } | ||
| break; | ||
| } |
There was a problem hiding this comment.
What happens if there's a slot other than Py_slot_invalid or Py_mod_exec here? Does _PySlotIterator_Next somehow handle that ahead-of-time?
There was a problem hiding this comment.
This is PyModule_ExecDef; it ignores slots other than Py_mod_exec.
Py_slot_invalid is the error indicator here (i.e. an exception was set in _PySlotIterator_Next) -- not the standard error API but nicer than the alternatives I tried.
| memcpy(tp_doc, it.current.sl_ptr, len); | ||
| } | ||
| break; |
There was a problem hiding this comment.
Same question here -- do we want a default: case for debugging?
There was a problem hiding this comment.
This is the first pass, most slots (especially the function pointers) are handled in the second pass.
_PySlotIterator_Next handles unknown slots, and ones that don't apply to types -- it sets an exception and reports Py_slot_invalid.
| _PySlot_KIND slot_struct_kind; | ||
| } _PySlotIterator_state; | ||
|
|
||
| #define SEEN_ENTRY_BITS (8 * sizeof(unsigned int)) |
There was a problem hiding this comment.
It doesn't seem like this is used outside slots.c. I think that it would make more sense to move it there, but if you'd really like to keep it in the header, let's be consistent and give this a _Py prefix.
Co-authored-by: Peter Bierma <zintensitydev@gmail.com> Co-authored-by: Petr Viktorin <encukou@gmail.com>
📚 Documentation preview 📚: https://cpython-previews--149055.org.readthedocs.build/