Merge branch 'main' into gh-148874/with-signal-exit-skip

Author: NekoAsakura

.github/workflows/posix-deps-apt.sh

@@ -26,9 +26,16 @@ apt-get -yq --no-install-recommends install \
     xvfb \
     zlib1g-dev
 
-# Workaround missing libmpdec-dev on ubuntu 24.04:
-# https://launchpad.net/~ondrej/+archive/ubuntu/php
-# https://deb.sury.org/
-sudo add-apt-repository ppa:ondrej/php
-apt-get update
-apt-get -yq --no-install-recommends install libmpdec-dev
+# Workaround missing libmpdec-dev on ubuntu 24.04 by building mpdecimal
+# from source. ppa:ondrej/php (launchpad.net) are unreliable 
+# (https://status.canonical.com) so fetch the tarball directly
+# from the upstream host.
+# https://www.bytereef.org/mpdecimal/
+MPDECIMAL_VERSION=4.0.1
+curl -fsSL "https://www.bytereef.org/software/mpdecimal/releases/mpdecimal-${MPDECIMAL_VERSION}.tar.gz" \
+    | tar -xz -C /tmp
+(cd "/tmp/mpdecimal-${MPDECIMAL_VERSION}" \
+    && ./configure --prefix=/usr/local \
+    && make -j"$(nproc)" \
+    && make install)
+ldconfig

Doc/c-api/perfmaps.rst

@@ -31,7 +31,7 @@ Note that holding an :term:`attached thread state` is not required for these API
    or ``-2`` on failure to create a lock. Check ``errno`` for more information
    about the cause of a failure.
 
-.. c:function:: int PyUnstable_WritePerfMapEntry(const void *code_addr, unsigned int code_size, const char *entry_name)
+.. c:function:: int PyUnstable_WritePerfMapEntry(const void *code_addr, size_t code_size, const char *entry_name)
 
    Write one single entry to the ``/tmp/perf-$pid.map`` file. This function is
    thread safe. Here is what an example entry looks like::

Doc/deprecations/pending-removal-in-3.20.rst

@@ -38,3 +38,8 @@ Pending removal in Python 3.20
   - :mod:`zlib`
 
   (Contributed by Hugo van Kemenade and Stan Ulbrych in :gh:`76007`.)
+
+* :mod:`ast`:
+
+  * Creating instances of abstract AST nodes (such as :class:`ast.AST`
+    or :class:`!ast.expr`) is deprecated and will raise an error in Python 3.20.

Doc/faq/programming.rst

@@ -1924,7 +1924,7 @@ correctly using identity tests:
 
    .. code-block:: python
 
-      _sentinel = object()
+      _sentinel = sentinel('_sentinel')
 
       def pop(self, key, default=_sentinel):
           if key in self:

Doc/howto/descriptor.rst

@@ -594,7 +594,7 @@ a pure Python equivalent:
 
     def object_getattribute(obj, name):
         "Emulate PyObject_GenericGetAttr() in Objects/object.c"
-        null = object()
+        null = sentinel('null')
         objtype = type(obj)
         cls_var = find_name_in_mro(objtype, name, null)
         descr_get = getattr(type(cls_var), '__get__', null)
@@ -1635,12 +1635,12 @@ by member descriptors:
 
 .. testcode::
 
-    null = object()
+    null = sentinel('null')
 
     class Member:
 
         def __init__(self, name, clsname, offset):
-            'Emulate PyMemberDef in Include/structmember.h'
+            'Emulate PyMemberDef in Include/descrobject.h'
             # Also see descr_new() in Objects/descrobject.c
             self.name = name
             self.clsname = clsname

Doc/howto/perf_profiling.rst

@@ -217,8 +217,9 @@ Example, using the :mod:`sys` APIs in file :file:`example.py`:
 How to obtain the best results
 ------------------------------
 
-For best results, Python should be compiled with
-``CFLAGS="-fno-omit-frame-pointer -mno-omit-leaf-frame-pointer"`` as this allows
+For best results, keep frame pointers enabled. On supported GCC-compatible
+toolchains, CPython builds itself with ``-fno-omit-frame-pointer`` and, when
+available, ``-mno-omit-leaf-frame-pointer`` by default. These flags allow
 profilers to unwind using only the frame pointer and not on DWARF debug
 information. This is because as the code that is interposed to allow ``perf``
 support is dynamically generated it doesn't have any DWARF debugging information

Doc/library/ast.rst

@@ -42,7 +42,7 @@ Node classes
 
 .. class:: AST
 
-   This is the base of all AST node classes.  The actual node classes are
+   This is the abstract base of all AST node classes.  The actual node classes are
    derived from the :file:`Parser/Python.asdl` file, which is reproduced
    :ref:`above <abstract-grammar>`.  They are defined in the :mod:`!_ast` C
    module and re-exported in :mod:`!ast`.
@@ -168,6 +168,15 @@ Node classes
    arguments that were set as attributes of the AST node, even if they did not
    match any of the fields of the AST node. These cases now raise a :exc:`TypeError`.
 
+.. deprecated-removed:: next 3.20
+
+    In the :ref:`grammar above <abstract-grammar>`, the AST node classes that
+    correspond to production rules with variants (aka "sums") are abstract
+    classes. Previous versions of Python allowed for the creation of direct
+    instances of these abstract node classes. This behavior is deprecated and
+    will be removed in Python 3.20.
+
+
 .. note::
     The descriptions of the specific node classes displayed here
     were initially adapted from the fantastic `Green Tree
@@ -271,18 +280,25 @@ Root nodes
 Literals
 ^^^^^^^^
 
-.. class:: Constant(value)
+.. class:: Constant(value, kind)
 
    A constant value. The ``value`` attribute of the ``Constant`` literal contains the
    Python object it represents. The values represented can be instances of :class:`str`,
    :class:`bytes`, :class:`int`, :class:`float`, :class:`complex`, and :class:`bool`,
    and the constants :data:`None` and :data:`Ellipsis`.
 
+   The ``kind`` attribute is an optional string. For string literals with a
+   ``u`` prefix, ``kind`` is set to ``'u'``. For all other
+   constants, ``kind`` is ``None``.
+
    .. doctest::
 
         >>> print(ast.dump(ast.parse('123', mode='eval'), indent=4))
         Expression(
             body=Constant(value=123))
+        >>> print(ast.dump(ast.parse("u'hello'", mode='eval'), indent=4))
+        Expression(
+            body=Constant(value='hello', kind='u'))
 
 
 .. class:: FormattedValue(value, conversion, format_spec)
@@ -2536,6 +2552,20 @@ and classes for traversing abstract syntax trees:
       Added the *color* parameter.
 
 
+.. function:: compare(a, b, /, *, compare_attributes=False)
+
+   Recursively compares two ASTs.
+
+   *compare_attributes* affects whether AST attributes are considered
+   in the comparison. If *compare_attributes* is ``False`` (default), then
+   attributes are ignored. Otherwise they must all be equal. This
+   option is useful to check whether the ASTs are structurally equal but
+   differ in whitespace or similar details. Attributes include line numbers
+   and column offsets.
+
+   .. versionadded:: 3.14
+
+
 .. _ast-compiler-flags:
 
 Compiler flags
@@ -2571,20 +2601,6 @@ effects on the compilation of a program:
    .. versionadded:: 3.8
 
 
-.. function:: compare(a, b, /, *, compare_attributes=False)
-
-   Recursively compares two ASTs.
-
-   *compare_attributes* affects whether AST attributes are considered
-   in the comparison. If *compare_attributes* is ``False`` (default), then
-   attributes are ignored. Otherwise they must all be equal. This
-   option is useful to check whether the ASTs are structurally equal but
-   differ in whitespace or similar details. Attributes include line numbers
-   and column offsets.
-
-   .. versionadded:: 3.14
-
-
 .. _ast-cli:
 
 Command-line usage

Doc/library/compression.zstd.rst

@@ -331,10 +331,14 @@ Compressing and decompressing data in memory
 
       If *max_length* is non-negative, the method returns at most *max_length*
       bytes of decompressed data. If this limit is reached and further
-      output can be produced, the :attr:`~.needs_input` attribute will
-      be set to ``False``. In this case, the next call to
+      output can be produced (or EOF is reached), the :attr:`~.needs_input`
+      attribute will be set to ``False``. In this case, the next call to
       :meth:`~.decompress` may provide *data* as ``b''`` to obtain
-      more of the output.
+      more of the output. The full content can thus be read like::
+
+        process_output(d.decompress(data, max_length))
+        while not d.eof and not d.needs_input:
+            process_output(d.decompress(b"", max_length))
 
       If all of the input data was decompressed and returned (either
       because this was less than *max_length* bytes, or because

Doc/library/email.policy.rst

@@ -403,11 +403,26 @@ added matters.  To illustrate::
    .. attribute:: utf8
 
       If ``False``, follow :rfc:`5322`, supporting non-ASCII characters in
-      headers by encoding them as "encoded words".  If ``True``, follow
-      :rfc:`6532` and use ``utf-8`` encoding for headers.  Messages
+      headers by encoding them as :rfc:`2047` "encoded words".  If ``True``,
+      follow :rfc:`6532` and use ``utf-8`` encoding for headers.  Messages
       formatted in this way may be passed to SMTP servers that support
       the ``SMTPUTF8`` extension (:rfc:`6531`).
 
+      When ``False``, the generator will raise
+      :exc:`~email.errors.HeaderWriteError` if any header includes non-ASCII
+      characters in a context where :rfc:`2047` does not permit encoded words.
+      This particularly applies to mailboxes ("addr-spec") with non-ASCII
+      characters, which can be created via
+      :class:`~email.headerregistry.Address`. To use a mailbox with a non-ASCII
+      domain name with ``utf8=False``, first encode the domain using the
+      third-party :pypi:`idna` or :pypi:`uts46` module or with
+      :mod:`encodings.idna`. It is not possible to use a non-ASCII username
+      ("local-part") in a mailbox when ``utf8=False``.
+
+      .. versionchanged:: 3.15
+         Can trigger the raising of :exc:`~email.errors.HeaderWriteError`.
+         (Earlier versions incorrectly applied :rfc:`2047` in certain contexts,
+         mostly notably in addr-specs.)
 
    .. attribute:: refold_source
 

Doc/library/faulthandler.rst

@@ -31,7 +31,8 @@ tracebacks:
 * Each string is limited to 500 characters.
 * Only the filename, the function name and the line number are
   displayed. (no source code)
-* It is limited to 100 frames and 100 threads.
+* It is limited to 100 frames per thread, and 100 threads
+  (configurable via *max_threads*).
 * The order is reversed: the most recent call is shown first.
 
 By default, the Python traceback is written to :data:`sys.stderr`. To see
@@ -55,16 +56,20 @@ at Python startup.
 Dumping the traceback
 ---------------------
 
-.. function:: dump_traceback(file=sys.stderr, all_threads=True)
+.. function:: dump_traceback(file=sys.stderr, all_threads=True, *, max_threads=100)
 
    Dump the tracebacks of all threads into *file*. If *all_threads* is
-   ``False``, dump only the current thread.
+   ``False``, dump only the current thread. *max_threads* caps the number
+   of threads dumped.
 
    .. seealso:: :func:`traceback.print_tb`, which can be used to print a traceback object.
 
    .. versionchanged:: 3.5
       Added support for passing file descriptor to this function.
 
+   .. versionchanged:: next
+      Added the *max_threads* keyword argument.
+
 
 Dumping the C stack
 -------------------
@@ -100,7 +105,7 @@ instead of the stack, even if the operating system supports dumping stacks.
 Fault handler state
 -------------------
 
-.. function:: enable(file=sys.stderr, all_threads=True, c_stack=True)
+.. function:: enable(file=sys.stderr, all_threads=True, c_stack=True, *, max_threads=100)
 
    Enable the fault handler: install handlers for the :const:`~signal.SIGSEGV`,
    :const:`~signal.SIGFPE`, :const:`~signal.SIGABRT`, :const:`~signal.SIGBUS`
@@ -116,6 +121,8 @@ Fault handler state
    traceback, unless the system does not support it. See :func:`dump_c_stack` for
    more information on compatibility.
 
+   *max_threads* caps the number of threads dumped when a fatal signal fires.
+
    .. versionchanged:: 3.5
       Added support for passing file descriptor to this function.
 
@@ -133,6 +140,9 @@ Fault handler state
    .. versionchanged:: 3.14
       The dump now displays the C stack trace if *c_stack* is true.
 
+   .. versionchanged:: next
+      Added the *max_threads* keyword argument.
+
 .. function:: disable()
 
    Disable the fault handler: uninstall the signal handlers installed by
@@ -146,15 +156,15 @@ Fault handler state
 Dumping the tracebacks after a timeout
 --------------------------------------
 
-.. function:: dump_traceback_later(timeout, repeat=False, file=sys.stderr, exit=False)
+.. function:: dump_traceback_later(timeout, repeat=False, file=sys.stderr, exit=False, *, max_threads=100)
 
    Dump the tracebacks of all threads, after a timeout of *timeout* seconds, or
    every *timeout* seconds if *repeat* is ``True``.  If *exit* is ``True``, call
    :c:func:`!_exit` with status=1 after dumping the tracebacks.  (Note
    :c:func:`!_exit` exits the process immediately, which means it doesn't do any
    cleanup like flushing file buffers.) If the function is called twice, the new
    call replaces previous parameters and resets the timeout. The timer has a
-   sub-second resolution.
+   sub-second resolution. *max_threads* caps the number of threads dumped.
 
    The *file* must be kept open until the traceback is dumped or
    :func:`cancel_dump_traceback_later` is called: see :ref:`issue with file
@@ -168,6 +178,9 @@ Dumping the tracebacks after a timeout
    .. versionchanged:: 3.7
       This function is now always available.
 
+   .. versionchanged:: next
+      Added the *max_threads* keyword argument.
+
 .. function:: cancel_dump_traceback_later()
 
    Cancel the last call to :func:`dump_traceback_later`.
@@ -176,11 +189,12 @@ Dumping the tracebacks after a timeout
 Dumping the traceback on a user signal
 --------------------------------------
 
-.. function:: register(signum, file=sys.stderr, all_threads=True, chain=False)
+.. function:: register(signum, file=sys.stderr, all_threads=True, chain=False, *, max_threads=100)
 
    Register a user signal: install a handler for the *signum* signal to dump
    the traceback of all threads, or of the current thread if *all_threads* is
    ``False``, into *file*. Call the previous handler if chain is ``True``.
+   *max_threads* caps the number of threads dumped.
 
    The *file* must be kept open until the signal is unregistered by
    :func:`unregister`: see :ref:`issue with file descriptors <faulthandler-fd>`.
@@ -190,6 +204,9 @@ Dumping the traceback on a user signal
    .. versionchanged:: 3.5
       Added support for passing file descriptor to this function.
 
+   .. versionchanged:: next
+      Added the *max_threads* keyword argument.
+
 .. function:: unregister(signum)
 
    Unregister a user signal: uninstall the handler of the *signum* signal