gh-145342: asyncio: Add guest mode for running inside external event loops

[congzhangzh]

Summary

Add asyncio.start_guest_run() which allows asyncio to run cooperatively
inside a host event loop (e.g. Tkinter, Qt, GTK). The host loop stays in
control of the main thread while asyncio I/O polling runs in a background
daemon thread.

Motivation

GUI applications with a native main loop (Tkinter, Qt, GTK) cannot use
asyncio.run() without blocking or replacing the host loop. Guest mode
enables incremental migration of GUI apps to async/await without replacing
the host event loop.

Implementation

• Add Lib/asyncio/guest.py with start_guest_run().
• Add three public methods to BaseEventLoop — poll_events(),
  process_events(), and process_ready() — that decompose _run_once()
  into independently callable steps (zero behavior change for existing code).
• Refactor _run_once() to delegate to the three new methods.
• Add comprehensive tests in Lib/test/test_asyncio/test_guest.py using a
  mock host loop (no GUI dependency, 12 test methods).
• Add a Tkinter demo in Doc/includes/asyncio_guest_tkinter.py.
• Add RST reference documentation in Doc/library/asyncio-guest.rst.
• Add NEWS entry.

Prior Art

Inspired by Trio's start_guest_run()
and the asyncio-guest proof-of-concept.

Testing

python -m pytest Lib/test/test_asyncio/test_guest.py -v

All 12 tests pass. The mock host loop tests cover: simple return, None return,
arguments, exceptions, cancellation from host, asyncio.sleep(), task creation,
asyncio.gather(), call_later, and call_soon_threadsafe.

[python-cla-bot]

All commit authors signed the Contributor License Agreement.

[congzhangzh]

@gvanrossum Hi Guido, I try to add guest mode to asyncio now, as I found that if I do not do it now, I will never have time to do it:)

[asvetlov]

How does the proposed approach work with the Windows proactor event loop?
Does the thread boundary crossing function well with IOCP ports?

[asvetlov]

A deamon thread smells like a red herring

[congzhangzh]

How does the proposed approach work with the Windows proactor event loop? Does the thread boundary crossing function well with IOCP ports?

It works in practice, but I agree it needs a careful check for Windows IOCP.

For the thread boundary, there is no concurrent access:

1. The main UI thread only triggers events and runs callbacks.
2. The backend thread completely suspends while the UI thread is active.
3. Result: Only one thread interacts with asyncio at any given moment.

This mutually design is based on Electron: https://www.electronjs.org/blog/electron-internals-node-integration

[congzhangzh]

BTW, the binary concept of a loop being 'running' or 'not running' breaks down a bit in guest mode. Internals like asyncio.sleep depend on it running, while other parts expect it stopped. We might need to adjust this abstraction.

[congzhangzh]

How does the proposed approach work with the Windows proactor event loop? Does the thread boundary crossing function well with IOCP ports?

It worked well in my past tests: https://github.com/congzhangzh/webview_python/tree/main/examples/async_with_asyncio_guest_run

Initially, I tried hooking directly into libuv or another event loop, but I later realized the event loop model is transparent to my solution.

Rather than relying on a standalone _run_once tick, the abstraction my solution actually depends on is select. For instance, the Windows IOCP proactor just relies on its internal implementation under the hood."

cpython/Lib/asyncio/base_events.py

Line 1977 in 6c417e4

def _run_once(self):

asyncio design just depend on the base High level _run_once?

cpython/Lib/asyncio/base_events.py

Line 2019 in 6c417e4

event_list = self._selector.select(timeout)

_run_once just depend on _select.select which is transparent for different implementation like select or IOCP

cpython/Lib/asyncio/windows_events.py

Line 444 in 6c417e4

def select(self, timeout=None):

Iocp select which depend on it's internal _poll

cpython/Lib/asyncio/windows_events.py

Line 762 in 6c417e4

def _poll(self, timeout=None):

_poll which depend on I/O completion ports

# windows_events.py
class IocpProactor:
    """Proactor implementation using IOCP."""
    # .. #
    def select(self, timeout=None):
        if not self._results:
            self._poll(timeout)
        tmp = self._results
        self._results = []
        try:
            return tmp
        finally:
            # Needed to break cycles when an exception occurs.
            tmp = None

    def _poll(self, timeout=None):
        # ...
        while True:
            status = _overlapped.GetQueuedCompletionStatus(self._iocp, ms)
            if status is None:
                break
            ms = 0
        # ...