Merge remote-tracking branch 'origin/main' into gh-159136-func-attrs

Author: nascheme

.github/dependabot.yml

@@ -117,15 +117,3 @@ updates:
         patterns: ["*"]
     cooldown:
       default-days: 14
-
-  - package-ecosystem: "pip"
-    target-branch: "3.13"
-    directory: "/Tools/"
-    schedule:
-      interval: "quarterly"
-    labels: ["skip issue", "skip news"]
-    groups:
-      pip:
-        patterns: ["*"]
-    cooldown:
-      default-days: 14

Doc/library/concurrent.interpreters.rst

@@ -224,11 +224,15 @@ This module defines the following functions:
    Initialize a new (idle) Python interpreter
    and return a :class:`Interpreter` object for it.
 
-.. function:: create_queue()
+.. function:: create_queue(maxsize=0)
 
    Initialize a new cross-interpreter queue and return a :class:`Queue`
    object for it.
 
+   *maxsize* is an integer that sets the upperbound limit on the number of
+   items that can be placed in the queue. Insertion will be blocked once this
+   size has been reached, until queue items are consumed. If *maxsize* is
+   less than or equal to zero, the queue size is infinite.
 
 Interpreter objects
 ^^^^^^^^^^^^^^^^^^^
@@ -268,7 +272,7 @@ Interpreter objects
       Some objects are actually shared and some are copied efficiently,
       but most are copied via :mod:`pickle`.  See :ref:`interp-object-sharing`.
 
-   .. method:: exec(code, /, dedent=True)
+   .. method:: exec(code, /)
 
       Run the given source code in the interpreter (in the current thread).
 
@@ -319,7 +323,9 @@ Communicating Between Interpreters
 .. class:: Queue(id)
 
    A wrapper around a low-level, cross-interpreter queue, which
-   implements the :class:`queue.Queue` interface.  The underlying queue
+   implements all the methods of :class:`queue.Queue` except for
+   :meth:`~queue.Queue.task_done`, :meth:`~queue.Queue.join`
+   and :meth:`~queue.Queue.shutdown`.  The underlying queue
    can only be created through :func:`create_queue`.
 
    Some objects are actually shared and some are copied efficiently,
@@ -331,17 +337,59 @@ Communicating Between Interpreters
 
       The queue's ID.
 
+   .. method:: qsize()
 
-.. exception:: QueueEmptyError
+      Return the number of items in the queue.
+
+   .. method:: empty()
+
+      Return ``True`` if the queue is empty, ``False`` otherwise.
+
+   .. method:: full()
+
+      Return ``True`` if the queue is full, ``False`` otherwise.
+
+   .. method:: put(obj, block=True, timeout=None)
+
+      Put *obj* into the queue.  If optional args *block* is true and
+      *timeout* is ``None`` (the default), block if necessary until a free
+      slot is available.  If *timeout* is a positive number, it blocks at
+      most *timeout* seconds and raises the :exc:`QueueFull` exception if
+      no free slot was available within that time.  Otherwise (*block* is
+      false), put an object on the queue if a free slot is immediately
+      available, else raise the :exc:`QueueFull` exception (*timeout* is
+      ignored in that case).
+
+   .. method:: put_nowait(obj)
+
+      Equivalent to ``put(obj, block=False)``.
+
+   .. method:: get(block=True, timeout=None)
+
+      Remove and return an object from the queue.  If optional args *block*
+      is true and *timeout* is ``None`` (the default), block if necessary
+      until an object is available.  If *timeout* is a positive number, it
+      blocks at most *timeout* seconds and raises the :exc:`QueueEmpty`
+      exception if no object was available within that time.  Otherwise
+      (*block* is false), return an object if one is immediately available,
+      else raise the :exc:`QueueEmpty` exception (*timeout* is ignored in
+      that case).
+
+   .. method:: get_nowait()
+
+      Equivalent to ``get(block=False)``.
+
+
+.. exception:: QueueEmpty
 
    This exception, a subclass of :exc:`queue.Empty`, is raised from
-   :meth:`!Queue.get` and :meth:`!Queue.get_nowait` when the queue
+   :meth:`Queue.get` and :meth:`Queue.get_nowait` when the queue
    is empty.
 
-.. exception:: QueueFullError
+.. exception:: QueueFull
 
    This exception, a subclass of :exc:`queue.Full`, is raised from
-   :meth:`!Queue.put` and :meth:`!Queue.put_nowait` when the queue
+   :meth:`Queue.put` and :meth:`Queue.put_nowait` when the queue
    is full.
 
 
@@ -382,3 +430,17 @@ Creating an interpreter and running code in it::
 
     t = interp.call_in_thread(run)
     t.join()
+
+Communicating between interpreters with a queue::
+
+    from concurrent import interpreters
+
+    interp = interpreters.create()
+    queue = interpreters.create_queue()
+
+    def worker(queue):
+        queue.put('spam!')
+
+    t = interp.call_in_thread(worker, queue)
+    print(queue.get())
+    t.join()

Doc/library/multiprocessing.rst

@@ -160,11 +160,21 @@ to start a process.  These *start methods* are
     No unnecessary resources are inherited.
 
     Available on POSIX platforms which support passing file descriptors over
-    Unix pipes such as Linux.  The default on those.
+    Unix pipes such as Linux.  The default on those, unless there is no
+    writable temporary directory (see :func:`tempfile.gettempdir`), which
+    the fork server requires; *spawn* is the default in that case.
 
     .. versionchanged:: 3.14
        This became the default start method on POSIX platforms.
 
+    .. versionchanged:: next
+       *spawn* is now the default start method when there is no writable
+       temporary directory, for example on a read-only file system.
+       Previously *forkserver* remained the default there and starting a
+       process failed with an :exc:`OSError`.  The default is determined
+       when it is first needed rather than when :mod:`!multiprocessing`
+       is imported.  See :gh:`155717`.
+
 .. versionchanged:: 3.4
    *spawn* added on all POSIX platforms, and *forkserver* added for
    some POSIX platforms.

Doc/library/tarfile.rst

@@ -1067,7 +1067,8 @@ can be:
 
 * the string ``'tar'``: Honor most *tar*-specific features (i.e. features of
   UNIX-like filesystems), but block features that are very likely to be
-  surprising or malicious. See :func:`tar_filter` for details.
+  surprising or malicious. Note that this filter does not limit the targets of
+  symbolic links. See :func:`tar_filter` for details.
 
 * the string ``'data'``: Ignore or block most features specific to UNIX-like
   filesystems. Intended for extracting cross-platform data archives.
@@ -1135,6 +1136,17 @@ reused in custom filters:
 
   Return the modified ``TarInfo`` member.
 
+  .. warning::
+     This filter does not check :attr:`TarInfo.linkname`. This means it does
+     not limit where a symbolic link member points. An absolute symbolic-link
+     target will remain absolute, and a relative target may resolve outside the
+     extraction destination. The destination path protection described above
+     concerns only where the archive member itself is written, not where a
+     symbolic link resolves. The stricter :func:`data_filter` rejects these
+     unsafe link targets, raising :class:`~tarfile.AbsoluteLinkError` for
+     absolute link targets or :class:`~tarfile.LinkOutsideDestinationError`
+     for link targets outside the destination.
+
   .. versionchanged:: next
 
      Filenames containing ``..`` components are now normalized.

Doc/tools/removed-ids.txt

@@ -93,6 +93,9 @@ using/windows.html: the-full-installer-deprecated
 using/windows.html: virtual-environments
 using/windows.html: windows-full
 
+# Duplicated label from reference/datamodel:
+using/windows.html: customization
+
 # Moved to library/stdtypes:
 reference/expressions.html: agen.__anext__
 reference/expressions.html: agen.aclose
@@ -114,3 +117,7 @@ library/turtle.html: changes-since-python-3-0
 # 'turtle' documentation reorganisation (gh-157847)
 library/turtle.html: note
 library/turtle.html: introduction
+
+# Wrong exception names in 'concurrent.interpreters' docs (gh-136563)
+library/concurrent.interpreters.html: concurrent.interpreters.QueueEmptyError
+library/concurrent.interpreters.html: concurrent.interpreters.QueueFullError

Doc/using/windows.rst

@@ -382,7 +382,6 @@ again. To completely clean up all Python runtimes, run with ``--purge`` before
 uninstalling the Python install manager.
 
 .. _pymanager-config:
-.. _customization:
 .. _customizing-default-python-versions:
 .. _customization-via-ini-files:
 .. _launcher-ini:

Lib/annotationlib.py

@@ -1105,9 +1105,17 @@ def type_repr(value):
 
     """
     if isinstance(value, (type, types.FunctionType, types.BuiltinFunctionType)):
-        if value.__module__ == "builtins":
+        module = value.__module__
+        if module is None:
+            # Built-in methods have __module__ set to None; recover the module
+            # from the bound object (the class itself for class-bound methods).
+            obj = getattr(value, "__self__", None)
+            if obj is not None:
+                cls = obj if isinstance(obj, type) else type(obj)
+                module = cls.__module__
+        if module is None or module == "builtins":
             return value.__qualname__
-        return f"{value.__module__}.{value.__qualname__}"
+        return f"{module}.{value.__qualname__}"
     elif isinstance(value, _Template):
         tree = _template_to_ast(value)
         return ast.unparse(tree)

Lib/multiprocessing/context.py

@@ -1,10 +1,10 @@
+import functools
 import os
 import sys
 import threading
 
 from . import process
 from . import reduction
-from . import util
 
 __all__ = ()
 
@@ -240,10 +240,19 @@ def _after_fork():
 class DefaultContext(BaseContext):
     Process = Process
 
-    def __init__(self, context):
-        self._default_context = context
+    def __init__(self, context=None):
+        if context is not None:
+            self._default_context = context
         self._actual_context = None
 
+    # The platform default is only determined when first needed, as doing
+    # so can be costly; importing multiprocessing must stay cheap.
+    # A cached_property rather than a property so that the attribute can
+    # still be assigned, deleted and patched like a plain one.
+    @functools.cached_property
+    def _default_context(self):
+        return _get_platform_default_context()
+
     def get_context(self, method=None):
         if method is None:
             if self._actual_context is None:
@@ -331,18 +340,21 @@ def _check_available(self):
     if reduction.HAVE_SEND_HANDLE:
         _concrete_contexts['forkserver'] = ForkServerContext()
 
-    # bpo-33725: running arbitrary code after fork() is no longer reliable
-    # on macOS since macOS 10.14 (Mojave). Use spawn by default instead.
-    # gh-84559: We changed everyones default to a thread safeish one in 3.14.
-    if (
-        reduction.HAVE_SEND_HANDLE
-        and sys.platform != 'darwin'
-        # gh-155717: forkserver requires to write temporary files
-        and util._has_writeable_tempdir()
-    ):
-        _default_context = DefaultContext(_concrete_contexts['forkserver'])
-    else:
-        _default_context = DefaultContext(_concrete_contexts['spawn'])
+    def _get_platform_default_context():
+        # bpo-33725: running arbitrary code after fork() is no longer reliable
+        # on macOS since macOS 10.14 (Mojave). Use spawn by default instead.
+        # gh-84559: We changed everyones default to a thread safeish one in 3.14.
+        if reduction.HAVE_SEND_HANDLE and sys.platform != 'darwin':
+            # gh-155717: forkserver requires to write temporary files.
+            #
+            # Imported here rather than at the top level as util is costly
+            # to import and is not otherwise needed to import this module.
+            from . import util
+            if util._has_writeable_tempdir():
+                return _concrete_contexts['forkserver']
+        return _concrete_contexts['spawn']
+
+    _default_context = DefaultContext()
 
 else:  # Windows
 
@@ -365,7 +377,10 @@ class SpawnContext(BaseContext):
     _concrete_contexts = {
         'spawn': SpawnContext(),
     }
-    _default_context = DefaultContext(_concrete_contexts['spawn'])
+    def _get_platform_default_context():
+        return _concrete_contexts['spawn']
+
+    _default_context = DefaultContext()
 
 #
 # Force the start method

Lib/multiprocessing/util.py

@@ -10,7 +10,6 @@
 import os
 import itertools
 import sys
-import tempfile
 import weakref
 import atexit
 import threading        # we want threading to install it's
@@ -155,7 +154,7 @@ def _remove_temp_dir(rmtree, tempdir):
         current_process._config['tempdir'] = None
 
 
-def _get_base_temp_dir():
+def _get_base_temp_dir(tempfile):
     """Get a temporary directory where socket files will be created.
 
     To prevent additional imports, pass a pre-imported 'tempfile' module.
@@ -216,8 +215,9 @@ def get_temp_dir():
     # get name of a temp directory which will be automatically cleaned up
     tempdir = process.current_process()._config.get('tempdir')
     if tempdir is None:
-        import shutil
-        base_tempdir = _get_base_temp_dir()
+        # Deferred until first use to keep importing this module cheap.
+        import shutil, tempfile
+        base_tempdir = _get_base_temp_dir(tempfile)
         tempdir = tempfile.mkdtemp(prefix='pymp-', dir=base_tempdir)
         info('created temp directory %s', tempdir)
         # keep a strong reference to shutil.rmtree(), since the finalizer
@@ -234,7 +234,14 @@ def _has_writeable_tempdir():
     #
     # See: https://github.com/python/cpython/issues/155717.
 
-    path = _get_base_temp_dir()
+    # Deferred until first use to keep importing this module cheap.
+    import tempfile
+
+    try:
+        path = _get_base_temp_dir(tempfile)
+    except OSError:
+        # No usable temporary directory at all.
+        return False
     if path is None:
         return False