Auditing a new main_thread_forkserver spawning backend#453
Auditing a new main_thread_forkserver spawning backend#453
main_thread_forkserver spawning backend#453Conversation
Brings `tractor`'s `pyproject.toml` (+ `uv.lock`) up to the
final state on the `subint_forkserver_backend` dev branch:
- `requires-python = ">=3.13, <3.15"` — stay on 3.13 floor
for working `stackscope` task-tree introspection;
3.14 still allowed for opt-in subint-family backends.
- New `[dependency-groups]`:
* `pytest-timeout>=2.3` — opt-in per-test cap (NOT used
as a global cap; see the long NOTE in
`[tool.pytest.ini_options]` for why pytest-timeout
breaks trio under fork-based backends).
* `psutil>=7.0.0` — used by `tractor._testing._reap` for
the `tractor-reap` zombie-actor + leaked-shm cleanup
utility (xplatform `Process.memory_maps`,
`Process.open_files`).
* `subints` group gated to `>=3.14` (`msgspec>=0.21.0`
for future PEP-684 isolated-subint work).
* `eventfd` + `sync_pause` gated to `>=3.13, <3.14`.
- `[tool.pytest.ini_options]` block now carries the
capture-mode + global-timeout tradeoffs as inline NOTEs.
Provenance: this is the squashed equivalent of ~7 commits
on `subint_forkserver_backend` between `9cf3d588` (initial
work) and `3c366cac` (drop global pytest-timeout cap). See
that branch for the full incremental history including the
`subint`-research dead-ends.
(this patch was generated in some part by [\`claude-code\`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Pulls the full conc-anal trail captured during the `subint_forkserver_backend` dev branch into `main`: - `subint_*_issue.md` (5 docs) — hang catalog + diagnosis for the explored `subint`-family backends. Several docs are referenced from the `_main_thread_forkserver.py` module docstring (CPython post-fork refusal, PEP 684 thread constraints, etc.) and need to be present for those links to resolve. - `subint_fork_blocked_by_cpython_post_fork_issue.md` — the canonical writeup of the CPython-level refusal that gates `os.fork()` from a sub-interpreter. Drives the design rationale for the main-thread-worker forkserver pattern. - `subint_fork_from_main_thread_smoketest.py` — standalone script empirically validating that `fork()` from a regular `threading.Thread` against the main interp works cleanly. Foundation for the variant-1 backend. - `fork_thread_semantics_execution_vs_memory.md` — reconciles trio's "leaked thread stacks" framing with the docstring's "every other thread is gone" framing as the memory-side vs execution-side of the same POSIX `fork()` reality. Provenance: squashed equivalent of ~10 commits on `subint_forkserver_backend` (`0f48ed2e`, `de4f470b`, `3ab99d55`, `4b5176e2`, `cf2e71d8`, `e3f4f5a3`, `c99d475d`, `f3cea714`, `4a325458`, `532a9834`). (this patch was generated in some part by [\`claude-code\`][claude-code-gh]) [claude-code-gh]: https://github.com/anthropics/claude-code
Captured Claude prompt I/O alongside the original `subint_forkserver_backend` dev work per the NLNet generative-AI-attribution policy (see `.claude/skills/prompt-io/`). Each pair of `.md`/`.raw.md` files records the prompt + response that drove a corresponding code commit on the source branch. Imported as a single squash so the historical provenance is preserved in `main`-bound history but without polluting the per-feature commit timeline. Provenance: squashed from ~4 commits on `subint_forkserver_backend` (`a65fded4`, `c041518b`, + inline addenda). (this patch was generated in some part by [\`claude-code\`][claude-code-gh]) [claude-code-gh]: https://github.com/anthropics/claude-code
Tooling configuration that came with the dev branch: - `.claude/skills/run-tests/SKILL.md` — extended with fork-spawn-aware test running guidance (zombie-actor cleanup, capture-mode tradeoffs, SIGINT-first cleanup ladder, lastfailed cache inspection). - `.claude/skills/conc-anal/SKILL.md` — minor refinements to the concurrency-analysis skill referenced by the new `ai/conc-anal/` docs. - `.claude/settings.local.json` — additional perms used by the new skills (`Skill(run-tests)`, `Skill(open-wkt)`, etc.) + `Read/Write(.claude/**)` for commit-msg authoring. - `.github/workflows/ci.yml` — pickup new dep groups + py-floor change. - `.gitignore` — add `notes/`, `snippets/`, `.claude/wkts/` patterns. Provenance: squashed from ~6 commits on `subint_forkserver_backend` (`9cf3d588`, `d3d6f646`, `ba86d482`, `b1a0753a`, `4106ba73`, `70d58c4b`, `d093c319`, `76d12060`). (this patch was generated in some part by [\`claude-code\`][claude-code-gh]) [claude-code-gh]: https://github.com/anthropics/claude-code
Standalone reaper utility used to scrub state left
behind by hard-killed actor sessions during
fork-spawn-backend hang triage:
- `scripts/tractor-reap` — CLI with `--zombies`
(reap orphaned `tractor._child` processes via
`psutil`) and `--shm` (sweep leaked
`multiprocessing.SharedMemory` segments backed by
dead pids).
- `tractor/_testing/_reap.py` — backing impl using
`psutil.Process.{memory_maps, open_files,
children}`. Layered so individual fixtures
(`reap_subactors_per_test` later) can call the
same primitives.
- `tractor/_testing/addr.py` — fix
`get_rando_addr()` cross-process collisions so
parallel pytest sessions don't fight over the same
random `reg_addr` ports / UDS sock paths.
Provenance: squashed from ~5 commits on
`subint_forkserver_backend` (`eae478f3`, `6d76b604`,
`4f12d69b`, `7c5dd4d0`).
(this patch was generated in some part by [\`claude-code\`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Plugin-level test infrastructure shared by both the
existing backends and the new fork-spawn one:
- `tractor/_testing/pytest.py`:
* Add `skipon_spawn_backend(<name>, reason=...)`
pytest marker. Lets per-test-file modules opt
out of specific backends with an explicit
reason string. Used heavily by the
cross-cutting test suite updates (Bucket H).
* `--enable-stackscope` CLI flag — installs the
`stackscope` SIGUSR1 handler at runtime gate
time WITHOUT pulling in the full `--tpdb`
debug-mode side effects. Pairs with
`tractor.devx._stackscope`'s file-tee +
run-sync-soon dispatch (Bucket E).
* `reap_subactors_per_test` fixture (opt-in) that
sweeps any `tractor._child` orphans whose
parent pid matches the current pytest process,
via the `tractor._testing._reap` helper.
* Backend-aware `--spawn-backend` option
parametrization.
- `tests/conftest.py`,
`tests/devx/conftest.py` — wire the new fixture
+ filter `start_method` allowlist for
pexpect-based debugger tests to include
`main_thread_forkserver`.
Provenance: squashed from ~6 commits on
`subint_forkserver_backend` (`3b26b59d`, `b376eb03`,
`f8178df0`, `5418f2dc`, `4c133ab5`, `d6e70e9d`).
(this patch was generated in some part by [\`claude-code\`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
The headline change: a new in-process forkserver that runs `os.fork()` from a regular `threading.Thread` attached to the main interpreter, sidestepping trio's classic fork-hostility (signal-wakeup-fd, epoll, threadpool caches) by isolating the syscall in a worker that has provably never entered trio. Spawn-method key: `'main_thread_forkserver'`. Reserves a sibling `'subint_forkserver'` key for the future variant-2 (subint-isolated child runtime, gated on jcrist/msgspec#1026 + PEP 684). Module layout (`tractor/spawn/`): - `_main_thread_forkserver.py` (NEW, 1024 LOC) — variant-1 IMPL. Includes the in-process forkserver worker thread, `_ForkedProc` `trio.Process`-shaped shim, post-fork fd scrubber (`_close_inherited_fds`), pidfd-based cancellable wait, plus the substantial module docstring covering POSIX `fork()` semantics ("execution-side gone" vs trio's "memory-side leaked" framing) + design rationale. - `_subint_forkserver.py` (NEW, 310 LOC) — variant-2 PLACEHOLDER. Today aliases dispatch to variant-1's proc spawner; reserved for the subint-hosted-trio child runtime. - `_subint.py` (NEW, 474 LOC) — variant-3 standalone subint backend. Wedged on cross-trio-instance hang upstream; carried as documented dead-code scaffold. - `_subint_fork.py` (NEW, 153 LOC) — variant-2 fork research. CPython-blocked (`_PyInterpreterState_DeleteExceptMain`); carried as a `NotImplementedError` stub pointing at the analysis doc. - `_spawn.py` — registers the four new spawn keys in `SpawnMethodKey` `Literal`, wires dispatch in `try_set_start_method` + `_methods` table. Runtime support: - `tractor/_child.py` — `_actor_child_main()` learns the `spawn_method` arg + post-fork `_state` reset. - `tractor/_root.py` — `open_root_actor` accepts `spawn_method=` + carries it through to actor init. - `tractor/runtime/_runtime.py` — relax the `enable_stack_on_sig` gate to honor `TRACTOR_ENABLE_STACKSCOPE` env-var (set by the pytest `--enable-stackscope` flag, Bucket D). - `tractor/runtime/_state.py` — refactor `_runtime_vars` into pure get/set API for clean post-fork reset; surface `spawn_method` to RPC log/repr paths. Devx: - `tractor/devx/_stackscope.py` — file-tee (`/tmp/tractor-stackscope-<pid>.log`) + `/dev/tty` writes that bypass pytest's stdio capture; `run_sync_soon`-based dispatch onto the trio loop so `stackscope.extract` recurses nursery children when called from a signal handler; suppress benign import-time `RuntimeWarning`s from `stackscope._glue`. - `tractor/devx/_debug_hangs.py` (NEW, 227 LOC) — hang-triage primitives (`dump_on_hang` etc.). IPC tweaks for fork-survival: - `tractor/ipc/_shm.py` — `SharedMemory` wedge fix under `main_thread_forkserver` (re-init posix primitive map post-fork). - `tractor/ipc/_mp_bs.py`, `tractor/ipc/_linux.py` — minor adjustments for fork-child IPC bringup. Tests (`tests/spawn/`): - `test_main_thread_forkserver.py` (652 LOC) — backend smoke + xfail draft for the orphaned-actor SIGINT cleanup behavior (xfail until issue #449 lands). - `test_subint_cancellation.py` (245 LOC) — variant-3 subint backend cancellation/teardown audit. Provenance: this is the squashed equivalent of ~30+ commits on `subint_forkserver_backend` between `82332fbc` (lift fork prims into a dedicated mod) and `2d4995e0` (route stackscope SIGUSR1 onto trio loop). See that branch for the full incremental history including the stepwise hang diagnoses, per-fix conc-anal annotations, and the variant-1/2 naming split (Phase D in the dev branch's history). Linked issues: - #379 — subint umbrella tracking issue - #449 — orphaned-subactor SIGINT cleanup hang (xfailed in the test suite) - #451 — Mode-A cancel-cascade hang - #452 — discovery-client `CLOSE_WAIT` fd leak (this patch was generated in some part by [\`claude-code\`][claude-code-gh]) [claude-code-gh]: https://github.com/anthropics/claude-code
Cross-cutting test-suite updates so the existing tests
work under the new `main_thread_forkserver` backend
alongside `trio` / `mp_*`:
- `skipon_spawn_backend(<name>, reason=...)` marks
added to tests that genuinely don't apply or are
known-broken under specific backends:
* `tests/test_pubsub.py` — `subint*`-family hangs
* `tests/test_shm.py` — `subint*` SharedMemory wedge
* `tests/test_inter_peer_cancellation.py` —
`subint*` cancellation leak
* `tests/discovery/test_registrar.py` —
`test_trynamic_trio` peer-discovery double-run
* `tests/test_cancellation.py` — known-failing
cases that need separate triage
- `reg_addr` standardization — replace ad-hoc port
hard-codings with the `reg_addr` fixture so
parallel pytest sessions don't collide
(`test_advanced_streaming`, `test_pubsub`,
`test_context_stream_semantics`,
`test_inter_peer_cancellation`,
`test_infected_asyncio`).
- Backend-aware timeouts — `maybe_expect_raises` in
`test_pldrx_limiting` and `test_dynamic_pub_sub`'s
`fail_after` cap honor `main_thread_forkserver`'s
per-spawn cost (30s vs 12s).
- `tests/devx/test_debugger.py` — extends the
`start_method` allowlist to enable pexpect-based
debugger tests under the fork backend (the forked
child runs its own `trio.run` exactly like the
`trio` backend so pexpect's process-boundary
expectations hold).
- `tests/test_ringbuf.py`, `tests/devx/test_tooling.py`,
`tests/devx/test_pause_from_non_trio.py` — minor
adjustments for fork-survival.
Provenance: squashed from ~15 commits on
`subint_forkserver_backend` (`b350aa09`, `2ca0f41e`,
`e3f4f5a3`, `1af21210`, `9b05f659`, `66f1941f`,
`44bdb169`, `aa3e2309`, `c99d475d`, `985ea76d`,
`f3cea714`, `4b2a0886`, `060f7d24`, `383b0fdd`,
`530160fa`, `65fcfbf2`, plus inline polish).
(this patch was generated in some part by [\`claude-code\`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
There was a problem hiding this comment.
Pull request overview
Mainlines the new main_thread_forkserver spawn backend (and related hardening) to enable fork-based spawning that is compatible with subinterpreter work, while adding diagnostics and test-harness safeguards for known hang classes.
Changes:
- Adds/threads new spawn backends (
main_thread_forkserver, stubs forsubint_fork/subint_forkserver) through the spawn registry and runtime. - Hardens teardown/cancellation paths (esp. fork-related deadlocks) and adds hang-diagnostic tooling + pytest harness improvements.
- Adjusts tests/fixtures and introduces
tractor-reapfor orphan subactor and leaked shm cleanup.
Reviewed changes
Copilot reviewed 59 out of 62 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
tractor/spawn/_subint_forkserver.py |
Adds reserved/stub backend and helper for running subints on worker threads. |
tractor/spawn/_subint_fork.py |
Adds subint_fork stub backend documenting CPython limitation. |
tractor/spawn/_spawn.py |
Registers new spawn method keys, gating, and dispatch table entries. |
tractor/runtime/_state.py |
Adds runtime-vars defaults/reset and a setter API used for fork-child state reset. |
tractor/runtime/_runtime.py |
Adjusts spawn-handshake handling; improves stackscope enablement; fixes fork-teardown deadlock and bounds peer shutdown wait. |
tractor/ipc/_shm.py |
Adjusts shm unlink behavior and makes teardown more robust to races. |
tractor/ipc/_mp_bs.py |
Disables multiprocessing.resource_tracker and forces SharedMemory(track=False) for fork-safety. |
tractor/ipc/_linux.py |
Refines cffi import error reporting for eventfd support. |
tractor/devx/_stackscope.py |
Improves SIGUSR1 stackscope dump reliability (tee to file/tty) and schedules dump on trio loop when possible. |
tractor/devx/_debug_hangs.py |
Adds reusable hang diagnostics (faulthandler-to-file + resource delta tracking). |
tractor/devx/__init__.py |
Re-exports new hang-diagnostic helpers. |
tractor/_testing/pytest.py |
Adds --enable-stackscope, skipon_spawn_backend marker, and orphan subactor reaping fixtures; improves typing. |
tractor/_testing/addr.py |
Reduces cross-process test port collisions by salting randomness with PID. |
tractor/_root.py |
Adds env-var overrides for loglevel/spawn method; expands debug-backend compatibility list. |
tractor/_child.py |
Refactors child entry into _actor_child_main for shared usage by multiple backends. |
tests/test_spawning.py |
Extends capture-related skip to fork-based backend. |
tests/test_shm.py |
Skips shm tests on subint backend (known hang class) and documents fork workaround expectations. |
tests/test_ringbuf.py |
Skips ringbuf tests by default; adds cffi import guard. |
tests/test_pubsub.py |
Adds skipon_spawn_backend('subint') marker for known hangs. |
tests/test_inter_peer_cancellation.py |
Adds skipon_spawn_backend('subint') marker for known hangs. |
tests/test_infected_asyncio.py |
Adds per-test reaping fixture usage and routes registries explicitly to reduce cascade failures. |
tests/test_context_stream_semantics.py |
Threads reg_addr into nurseries to isolate runs and reduce interference. |
tests/test_cancellation.py |
Adds backend skips/typing tweaks; adds timeouts in some cases and routes reg_addr into nurseries. |
tests/test_advanced_streaming.py |
Reworks cancellation expectations and avoids pytest-timeout for fork backends (uses trio timeouts). |
tests/spawn/test_subint_cancellation.py |
Adds subint cancellation audit tests with hang dumps. |
tests/msg/test_pldrx_limiting.py |
Makes timeouts backend-aware to reduce fork-backend flakiness. |
tests/discovery/test_registrar.py |
Adds hang dump and backend skip notes for known subint hang class. |
tests/devx/test_tooling.py |
Skips greenback-dependent tooling tests when greenback isn’t installed. |
tests/devx/test_pause_from_non_trio.py |
Skips greenback-dependent pause tests when greenback isn’t installed. |
tests/devx/test_debugger.py |
Adjusts timeout usage for debugger test. |
tests/devx/conftest.py |
Uses env vars to select spawn backend/loglevel for example-script runs; expands supported spawners. |
tests/conftest.py |
Adds typing to fixtures touched by harness changes. |
scripts/tractor-reap |
Introduces CLI for orphan subactor reap and optional /dev/shm sweep. |
pyproject.toml |
Updates Python support range; reorganizes dependency groups; sets pytest defaults (notably --capture=sys). |
.github/workflows/ci.yml |
Adjusts pytest invocation formatting and capture mode override. |
.gitignore |
Expands ignore rules for local/AI tooling artifacts. |
ai/prompt-io/claude/20260422T200723Z_797f57c_prompt_io.raw.md |
Adds provenance log (raw). |
ai/prompt-io/claude/20260422T200723Z_797f57c_prompt_io.md |
Adds provenance log (summary). |
ai/prompt-io/claude/20260420T192739Z_5e8cd8b2_prompt_io.raw.md |
Adds provenance log (raw). |
ai/prompt-io/claude/20260420T192739Z_5e8cd8b2_prompt_io.md |
Adds provenance log (summary). |
ai/prompt-io/claude/20260418T042526Z_26fb820_prompt_io.raw.md |
Adds provenance log (raw). |
ai/prompt-io/claude/20260418T042526Z_26fb820_prompt_io.md |
Adds provenance log (summary). |
ai/prompt-io/claude/20260417T124437Z_5cd6df5_prompt_io.raw.md |
Adds provenance log (raw). |
ai/prompt-io/claude/20260417T124437Z_5cd6df5_prompt_io.md |
Adds provenance log (summary). |
ai/conc-anal/subint_forkserver_thread_constraints_on_pep684_issue.md |
Adds analysis/tracking doc for future constraints cleanup. |
ai/conc-anal/subint_forkserver_mp_shared_memory_issue.md |
Adds analysis/post-mortem doc for shm/resource_tracker fork issues. |
ai/conc-anal/subint_fork_from_main_thread_smoketest.py |
Adds CPython-level smoke test for the main-thread worker fork approach. |
ai/conc-anal/subint_fork_blocked_by_cpython_post_fork_issue.md |
Adds analysis doc for CPython post-fork refusal from subinterpreters. |
ai/conc-anal/subint_cancel_delivery_hang_issue.md |
Adds analysis doc for Ctrl-C-able parent-side hang after subint teardown. |
ai/conc-anal/fork_thread_semantics_execution_vs_memory.md |
Adds explanatory doc on fork semantics in multithreaded programs. |
.claude/skills/conc-anal/SKILL.md |
Adds internal guidance on teardown waits and capture-pipe hang pattern. |
.claude/settings.local.json |
Updates local tooling permissions/config. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| 'XXX SUBINT HANGING TEST XXX\n' | ||
| 'See oustanding issue(s)\n' | ||
| # TODO, put issue link! | ||
| ) |
| reason=( | ||
| 'XXX SUBINT HANGING TEST XXX\n' | ||
| 'See oustanding issue(s)\n' | ||
| # TODO, put issue link! | ||
| ) |
| infect_asyncio: bool = False, | ||
| task_status: TaskStatus[Portal] = trio.TASK_STATUS_IGNORED, | ||
| proc_kwargs: dict[str, any] = {}, | ||
|
|
| > **Status:** placeholder. Today | ||
| > `--spawn-backend=subint_forkserver` aliases to | ||
| > `main_thread_forkserver_proc` (variant 1, see | ||
| > `tractor.spawn._main_thread_forkserver`). A follow-up commit | ||
| > in this PR series flips the alias to a `NotImplementedError` | ||
| > stub reserving the `'subint_forkserver'` key for the literal |
| infect_asyncio: bool = False, | ||
| task_status: TaskStatus[Portal] = trio.TASK_STATUS_IGNORED, | ||
| proc_kwargs: dict[str, any] = {}, | ||
|
|
| # is honored inside `tractor._root.open_root_actor()`, | ||
| # so no per-script edits are required. | ||
| 'main_thread_forkserver', | ||
| 'subint_forkserver', |
| --capture=fd | ||
| # ^XXX^ can't work with --spawn-method=main_thread_forkserver |
| 'XXX SUBINT GIL-CONTENTION HANGING TEST XXX\n' | ||
| 'See oustanding issue(s)\n' | ||
| # TODO, put issue link! | ||
| ) |
| try: | ||
| import cffi | ||
| except ImportError as ie: | ||
| if sys.version_info < (3, 14): | ||
| ie.add_note( | ||
| f'The `cffi` pkg has no 3.14 support yet.\n' | ||
| ) | ||
|
|
| # `TRACTOR_LOGLEVEL` env-var wins over any caller-passed | ||
| # `loglevel` so devs/test-runs can crank (or silence) | ||
| # console verbosity without touching application code. | ||
| env_ll_report: str = '' | ||
| if env_ll := os.environ.get('TRACTOR_LOGLEVEL'): | ||
| loglevel = env_ll | ||
| env_ll_report: str = ( | ||
| f'Detected env-var setting,\n' | ||
| f'TRACTOR_LOGLEVEL={env_ll!r}\n' | ||
| f'\n' | ||
| f'Setting console loglevel per,\n' | ||
| f'loglevel={loglevel!r}\n' | ||
| ) | ||
| if ( | ||
| loglevel | ||
| and | ||
| loglevel.upper() != env_ll.upper() | ||
| ): | ||
| env_ll_report += ( | ||
| f'\n' | ||
| f'NOTE env-var OVERRIDES caller-passed,\n' | ||
| f'loglevel={loglevel!r}\n' | ||
| ) | ||
|
|
||
| loglevel: str = ( | ||
| loglevel | ||
| or | ||
| log._default_loglevel | ||
| ) | ||
| loglevel: str = loglevel.upper() | ||
|
|
||
| assert loglevel | ||
| _log = log.get_console_log( | ||
| level=loglevel, | ||
| name='tractor', | ||
| logger=logger, | ||
| ) | ||
| assert _log | ||
| if env_ll_report: | ||
| _log.info(env_ll_report) | ||
|
|
||
| # `TRACTOR_SPAWN_METHOD` env-var wins over any caller-passed | ||
| # `start_method` so devs/test-runs can swap the actor spawn | ||
| # backend without touching application code (e.g. driving | ||
| # the `examples/debugging/<script>.py` suite under each | ||
| # backend from `tests/devx/conftest.py::spawn`). | ||
| if env_sm := os.environ.get('TRACTOR_SPAWN_METHOD'): | ||
| start_method: str = env_sm | ||
| env_sm_report: str = ( | ||
| f'Detected env-var setting,\n' | ||
| f'TRACTOR_SPAWN_METHOD={env_sm!r}\n' | ||
| f'\n' | ||
| f'Setting spawn backend as,\n' | ||
| f'start_method={env_sm!r}\n' | ||
| ) | ||
| if ( | ||
| start_method | ||
| and | ||
| start_method != env_sm | ||
| ): | ||
| _log.warning( | ||
| env_sm_report | ||
| + | ||
| f'NOTE env-var OVERRIDES caller-passed,\n' | ||
| f'`start_method={start_method!r}`\n' | ||
| ) | ||
| else: | ||
| _log.info(env_sm_report) |
|
Heh, looking through these commits, i actually think they're a bit too chunky 😂 🫣 .. so i might replace this with something else FYI. either way gonna use this one to get CI green.. |
main_thread_forkserver spawning backendmain_thread_forkserver spawning backend
|
Yeah, so much work since thinking this was near stable, and in hindsight i don't like losing the granular commit history from the very iterative work getting our test suite stable against this spawner. Closing. |
Re-worked/squashed history from #447 but focused on promoting and mainlining the (nearly CI audited as working)
tractor.spawn._main_thread_forkserverbackend tinkered from the histories (commit-chronologically) of,os.fork()spawning backend #448Surprisingly (at least to me) this patch set was mostly vibe coded with much human intuition/guidance to get us to what seems to be a much superior approach to the stdlib's
multiprocessing"forkserver" spawn method,which has been very problematic for us within rigorous "SC safe" constraints and more generally for nested subactor trees per,
Differences from #447 in this patch
despite the diff being identical this instead,
focuses/emphasizes the 3.13+ and-already-working
.spawn._main_thread_forkerserbackend despite bringing in the code first prototyped in thespawn._subint,spawn._subint_forkandspawn._subint_forkserverbackends each first presented in the draft PRs above.minimized set of larger commits reworked by
claudeto make the history by commit more immediately obvious, high-level-grokable for human onlookers.will ensure we get a clean CI run for all of,
--spawn-backend=main_thread_forkserver+--tpt-proto=[tcp|uds]detail out various new
pytestharness features developed while prototyping this backend and debugging various complex testing edge cases including some very difficult to decipher test-suite hangs on the new backend.Still TODO before this lands,