Skip to content

gh-149044: Implement PEP 820 – PySlot: Unified slot system for the C API#149055

Merged
encukou merged 54 commits intopython:mainfrom
encukou:slots-pep820-mrg
May 5, 2026
Merged

gh-149044: Implement PEP 820 – PySlot: Unified slot system for the C API#149055
encukou merged 54 commits intopython:mainfrom
encukou:slots-pep820-mrg

Conversation

@encukou
Copy link
Copy Markdown
Member

@encukou encukou commented Apr 27, 2026

Copy link
Copy Markdown
Member

@ZeroIntensity ZeroIntensity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Documentation looks pretty good. I haven't looked at all of the C code yet.

Comment thread Doc/c-api/slots.rst Outdated
Comment thread Doc/c-api/slots.rst Outdated
@AlexWaygood AlexWaygood removed their request for review April 28, 2026 14:06
@brettcannon brettcannon removed their request for review April 28, 2026 16:44
Comment thread Doc/c-api/slots.rst Outdated
encukou and others added 2 commits April 30, 2026 12:07
@read-the-docs-community
Copy link
Copy Markdown

read-the-docs-community Bot commented Apr 30, 2026

Copy link
Copy Markdown
Contributor

@ngoldbaum ngoldbaum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wrote FFI bindings for this PR for PyO3 this afternoon and spotted several minor issues.

Comment thread Include/slots.h
Comment on lines +48 to +52
#define PySlot_PTR(NAME, VALUE) \
{(NAME), PySlot_INTPTR, {0}, {(void*)(VALUE)}}

#define PySlot_PTR_STATIC(NAME, VALUE) \
{(NAME), PySlot_INTPTR | PySlot_STATIC, {0}, {(void*)(VALUE)}}
Copy link
Copy Markdown
Contributor

@ngoldbaum ngoldbaum May 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you spell these last two with the explicit names of the struct fields like the ones above? Also while you're touching this code, IMO these two definitions should come before PySlot_END.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These macros are for code compatible with C++11 and below, which doesn't support designated initializers (that is, the explicit names).

So, both style issues are intentional :) I added a comment.

Comment thread Include/slots.h Outdated
Comment thread Include/slots.h
#define PySlot_STATIC 0x02
#define PySlot_INTPTR 0x04

#define Py_slot_invalid 0xffff
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for consistency should this be called Py_slot_INVALID maybe?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would be consistent with the wrong thing. Slot IDs are named Py_tp_repr, Py_mod_exec, etc. Flags are a new thing, so they follow the current style guide: mixed prefix + uppercase.

Comment thread Include/slots_generated.h
#define _Py_SLOT_COMPAT_VALUE(OLD, NEW) OLD
#endif

#define Py_slot_end 0
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe worth adding a comment above these that they're all u16 values?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In C, the preprocessor macros are just numbers :)
In PyType_Slot, these values are used as int.

Comment thread Include/object.h Outdated
Comment thread Include/slots_generated.h
#define Py_tp_finalize 80
#define Py_am_send 81
#define Py_tp_vectorcall 82
#define Py_tp_token 83
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything through here is duplicated with typeslots.h. Was that intentional? It probably makes sense for there to be only one source of truth for these constants.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the catch! I seem to have dropped the commit that removes the unused file.

Comment thread Include/slots.h
#ifndef _Py_HAVE_SLOTS_H
#define _Py_HAVE_SLOTS_H

typedef void (*_Py_funcptr_t)(void);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PyO3 needs to wrap this type directly so you might as well give this a public non-underscored name.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can PyO3 use the underlying type, or better yet, some “generic C function pointer” type?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately no, because Rust doesn't let you cast between function pointer types without using unsafe and std::mem::transmute. That said, in my second draft of this, I isolated the unsafety into a PySlot_FUNC rust macro that hides the wrapped function pointer, so I take back my comment from last night.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keep in mind this is not the correct function type, just a function type. At this level, unsafe transmuation is exactly what you need in Rust.
(A safer layer on top of the ABI would need to have separate C-specific and C++-specific variants, and presumably a Rust-specific one as well. I didn't get that into 3.15.)

@encukou
Copy link
Copy Markdown
Member Author

encukou commented May 4, 2026

I intend to merge this today (or early tomorrow) to get it in 3.15, as postponing to 3.16 would be very inconvenient. Beta is for fixing features (or removing, if this turns out to be a bad idea after all).

Copy link
Copy Markdown
Member

@ZeroIntensity ZeroIntensity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is in a mergeable state. I read through all of the PEP, documentation, and code, and I couldn't find any glaring issues.

I left a number of non-blocking questions/comments; feel free to ignore them for now and address them after the beta freeze.

Comment thread Doc/c-api/slots.rst Outdated
Comment thread Include/internal/pycore_slots.h Outdated
Comment thread Include/internal/pycore_slots.h Outdated
Comment thread Include/slots.h Outdated
Comment on lines 925 to 929
self.assertRaisesRegex(
SystemError,
"Py_tp_bases is not a tuple",
TypeError,
"metaclass conflict",
_testcapi.create_heapctype_with_none_bases_slot
)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, this seems like a downgrade in terms of the error message. I don't see a change to create_heapctype_with_none_bases_slot in this PR, so is there a reason the error changed?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's a side effect from merging Py_tp_base & Py_tp_bases -- the latter strictly required a tuple of types, so it had this message.
This is a silly restriction. I'll document lifting it.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(And I'll get SC permission to update the PEP, or revert this.)

Comment thread Objects/moduleobject.c Outdated
Comment thread Objects/moduleobject.c
Comment on lines +794 to 796
}
break;
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if there's a slot other than Py_slot_invalid or Py_mod_exec here? Does _PySlotIterator_Next somehow handle that ahead-of-time?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is PyModule_ExecDef; it ignores slots other than Py_mod_exec.

Py_slot_invalid is the error indicator here (i.e. an exception was set in _PySlotIterator_Next) -- not the standard error API but nicer than the alternatives I tried.

Comment thread Objects/typeobject.c
Comment on lines +5403 to 5405
memcpy(tp_doc, it.current.sl_ptr, len);
}
break;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question here -- do we want a default: case for debugging?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the first pass, most slots (especially the function pointers) are handled in the second pass.

_PySlotIterator_Next handles unknown slots, and ones that don't apply to types -- it sets an exception and reports Py_slot_invalid.

Comment thread Tools/c-analyzer/c_common/scriptutil.py Outdated
Comment thread Include/internal/pycore_slots.h Outdated
_PySlot_KIND slot_struct_kind;
} _PySlotIterator_state;

#define SEEN_ENTRY_BITS (8 * sizeof(unsigned int))
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't seem like this is used outside slots.c. I think that it would make more sense to move it there, but if you'd really like to keep it in the header, let's be consistent and give this a _Py prefix.

encukou and others added 5 commits May 4, 2026 22:58
@encukou encukou added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label May 4, 2026
@encukou encukou merged commit 508b498 into python:main May 5, 2026
93 of 103 checks passed
@encukou encukou deleted the slots-pep820-mrg branch May 5, 2026 07:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🔨 test-with-buildbots Test PR w/ buildbots; report in status section

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants