docs: Stdcoroutine-0: Boost.Coroutine to C++20 std::coroutine migration plan

[pratikmankawde]

Github PR doesn't format this doc very well. So, feel free to read it on Github branch page.

High Level Overview of Change

Adds a comprehensive migration plan document (BoostToStdCoroutineSwitchPlan.md) and task list (BoostToStdCoroutineTaskList.md) for switching rippled from Boost.Coroutine2 (stackful) to C++20 standard coroutines (stackless).

This is PR 0 in the StdCoroutineSwitch chain — it contains only the plan and task-list documents, no code changes.

PR Chain

Implementation is in draft mode. Still evolving and being reviewed.

PR	Branch	Description
#6643 (this)	pratik/Swtich-to-std-coroutines → develop	Migration plan + task list
#6421	pratik/std-coro/add-coroutine-primitives → develop	CoroTask, CoroTaskRunner, JobQueueAwaiter primitives
#6423	pratik/std-coro/migrate-entry-points → add-coroutine-primitives	Migrate HTTP/WS/gRPC entry points to postCoroTask(); remove RPC::Context::coro
#6428	pratik/std-coro/migrate-test-code → migrate-entry-points	Migrate coroutine tests to C++20 API
#6429	pratik/std-coro/cleanup-boost-coroutine → migrate-test-code	Remove Boost::coroutine dependency and legacy Coro API; keep Boost::context for boost::asio::spawn
#6525	pratik/std-coro/tsan-fixes → cleanup-boost-coroutine	TSAN data-race fixes for CoroTaskRunner

Context of Change

The plan covers:

• Research & viability analysis — why C++20 stackless coroutines work for rippled's shallow yield pattern
• Current state audit — all coroutine touchpoints, JobQueue::Coro internals, entry points, handlers
• Migration strategy — incremental 4-phase approach with coexistence period
• Implementation design — CoroTask<T>, CoroTaskRunner (core lifecycle primitive), yieldAndPost() (compiler-robust inline awaiter), JobQueueAwaiter (deprecated single-shot awaiter), API mapping
• CMake / conan outcome — Boost::coroutine removed, Boost::context retained for boost::asio::spawn, sanitizer defines (BOOST_USE_ASAN / BOOST_USE_TSAN / BOOST_USE_UCONTEXT) added
• RipplePathFind design — 30-second std::condition_variable bounded wait (trade-off documented; PathFindAwaiter listed as a follow-up)
• Testing & validation — unit tests, sanitizer testing (ASAN/TSAN), benchmarks, regression methodology
• Risks & mitigation — risk matrix, rollback strategy, stackful→stackless limitation analysis
• Standards & guidelines — coroutine design rules, thread safety, naming conventions, code review checklist
• Follow-ups — PathFindAwaiter, grpc::ServerContext::IsCancelled wiring, migrating boost::asio::spawn callers

API Impact

• Public API: New feature (new methods and/or new fields)
• Public API: Breaking change (in general, breaking changes should only impact the next api_version)
• libxrpl change (any change that may affect libxrpl or dependents of libxrpl)
• Peer protocol change (must be backward compatible or bump the peer protocol version)

[xrplf-ai-reviewer]

Three issues flagged inline: a year typo in the document header, and two design gaps in the gRPC migration path — CallData lifetime analysis is missing (potential use-after-free), and ServerContext cancellation propagation is unaddressed for suspended coroutines.

Review by Claude Opus 4.6 · Prompt: V12

[xrplf-ai-reviewer]

Plan gap: CallData ownership chain under C++20 not analyzed — potential use-after-free risk.

In the Boost model, shared_ptr<Coro> inside the lambda ensures CallData outlives the coroutine. With C++20, if the gRPC completion queue fires and destroys CallData while the coroutine frame still holds a reference (via RPC::Context), this is a use-after-free — the exact dangling reference risk from Concern 5, but unaddressed for the gRPC code path.

Suggested addition in Milestone 2, task 2.3: Explicitly audit CallData object lifetime relative to the CoroTaskRunner frame. Ensure CallData is kept alive (e.g., via shared_from_this() or explicit capture) for the full coroutine duration. Add a TSAN/ASAN test specifically for gRPC request lifetime.

See: gRPC

[xrplf-ai-reviewer]

Plan gap: gRPC ServerContext cancellation propagation not addressed in the migration.

When CallData::process() is migrated to postCoroTask() in Phase 2 (task 2.3), there is no discussion of what happens if the gRPC client disconnects or times out while the coroutine is suspended (e.g., during pathfinding). The coroutine will resume on the JobQueue with no awareness of cancellation — wasting resources and potentially writing to a dead stream.

Suggested addition in Phase 2, task 2.3: Document whether grpc::ServerContext* is threaded through RPC::Context. If so, add a cancellation check in JobQueueAwaiter::await_suspend() or at the co_await resume point: if grpc_context->IsCancelled(), return codes.Canceled rather than continuing into the handler body.

See: gRPC

[xrplf-ai-reviewer]

Two off-by-one bugs in BUILD.md bash loops and a likely year typo in the plan doc — see inline comments.

Review by Claude Opus 4.6 · Prompt: V12

[xrplf-ai-reviewer]

Five issues flagged inline: a year typo in the document header, and four architectural gaps in the gRPC migration coverage — missing streaming RPC audit, missing CompletionQueue lifecycle analysis, missing shutdown handling in task 2.3, and a high-severity regression where the FAQ documents a 30-second thread-blocking synchronous wait that contradicts the migration's core goals.

Review by Claude Opus 4.6 · Prompt: V12

[xrplf-ai-reviewer]

Section 5.4 lists only the unary gRPC entry point — streaming RPC handlers are not audited. Add a paragraph to Section 5.4 confirming: (a) which rippled proto methods are unary vs streaming, (b) whether any streaming handler calls postCoro() or yield(), and (c) whether streaming handlers use a separate code path unaffected by this migration. Without this, a streaming RPC could silently retain the old Boost path after Phase 4 cleanup removes Coro.

See: context | gRPC

[xrplf-ai-reviewer]

No design notes on CallData lifecycle with gRPC's CompletionQueue. The plan identifies GRPCServer.cpp:102 as an entry point but doesn't verify that CoroTaskRunner lifetime outlives all CompletionQueue callbacks that reference it, or that coroutine frame ownership is safe across tag firings. Add an analysis tracing: CQ tag posted → process() called → coroutine suspended → CQ tag fires again → coroutine resumed, and confirm no raw coroutine_handle<> is stored in CQ tags without RAII ownership.

See: gRPC

[xrplf-ai-reviewer]

Task 2.3 is missing gRPC shutdown handling. The old Coro::post() returned false when the JobQueue was stopping, letting the CallData handler detect shutdown and call Finish() with an appropriate status. Add a sub-task: Verify that when addJob() returns false during shutdown, the awaiter causes the coroutine to terminate and the gRPC call is finished with grpc::StatusCode::UNAVAILABLE. Write a test that shuts down the JobQueue while a gRPC coroutine is suspended and confirms no RPC hangs indefinitely.

See: gRPC

[xrplf-ai-reviewer]

The FAQ admits blocking a worker thread for up to 30 seconds via std::condition_variable, directly contradicting the migration's goal of freeing threads during suspension and voiding the performance gains claimed in Section 4.4 for this code path. Either implement PathFindAwaiter (task 3.2) to properly suspend the coroutine, or at minimum document this as a known regression and ensure the pathfinding timeout is capped below the gRPC deadline so the thread is guaranteed to be released before the client times out.

See: std::condition_variable | gRPC

[a1q123456]

Given that our coroutine use case isn't different from others at all, I think we can use the existing coroutine implementation in boost.asio instead of reinventing the wheel and implementing our own promise and future types.

This approach gives us some benefits:

1. Properly tested coroutine implementation - boost.asio is used everywhere and by out sourcing this part to boost, we don't need to scratch our heads looking at coroutine-related and concurrent-related bugs
2. We get less code and it shortens development time
3. Crystal-clear design - co_spawn schedules a coroutine on an executor; boost::asio::detached means fire and forget, boost::asio::use_future returns a std::future so that you can wait for it synchronously

To make it work, we'll need to implement an executor that meets asio's requirement, refactor JobQueue and Coro to use the executor. As the second phase, we refactor to use boost.asio coroutine, and then we can replace Workers with boost::thread_pool.

I propose this plan:

Phase 1:

1. Implement JobQueueExecutor — Custom Asio executor with execute() that calls addJob(), carrying a JobType for priority. Wraps the function to save/restore LocalValues pointer around invocation.
2. Refactor JobQueue and Coro to use JobQueueExecutor
3. Unit tests for JobQueueExecutor. The public API of JobQueue shouldn't change at this moment, we only need to ensure the current JobQueue tests pass.

Phase 2: Replace Coro with C++20 coroutines

1. Remove Coro class and replace postCoro — Delete Coro, Coro.ipp, Coro_create_t, nSuspend_. New method uses co_spawn + JobQueueExecutor
2. Update RPC::Context — Replace std::shared_ptrJobQueue::Coro coro member with a new mechanism (e.g. the executor, or a yield awaitable)
3. Update ServerHandler (HTTP RPC) — 3 call sites: onRequest(), processSession(Session), processRequest()
4. Update ServerHandler (WebSocket) — 1 call site: onWSMessage() + processSession(WSSession)
5. Update GRPCServer — 1 call site: CallData::onDone() + CallData::process()
6. Update RipplePathFind — Most complex: yield()/post()/resume() pattern with async callback needs replacing with co_await on a promise/event
7. Update existing tests — Coroutine_test.cpp, JobQueue_test.cpp
8. We may want to rename Coro to something like CoroutineHandle or CoroutineFrame or whatever
9. Unit tests for the new JobQueue and Coro

Phase 3: Replace Workers with boost::asio::thread_pool

1. Replace Workers internals — Remove Workers class, replace with thread_pool. Remove Workers::Callback inheritance from JobQueue. Drop setNumberOfThreads.
2. Clean up Workers tests

[Copilot]

Pull request overview

Adds documentation to guide the (now-implemented) migration from Boost.Coroutine2 stackful coroutines to C++20 std::coroutine stackless coroutines in rippled, plus minor spellchecker dictionary updates to support the new docs.

Changes:

• Add a comprehensive migration plan document with architecture analysis, phased rollout, testing strategy, and guidelines.
• Add a milestone/task checklist companion document for tracking the migration work.
• Update cspell dictionary with coroutine/migration terminology and proper nouns used in the docs.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File	Description
cspell.config.yaml	Adds new allowed words referenced by the added migration docs.
BoostToStdCoroutineTaskList.md	New milestone-by-milestone task checklist for the coroutine migration effort.
BoostToStdCoroutineSwitchPlan.md	New detailed migration plan, background, risk analysis, testing/validation strategy, and guidelines.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The header says "Status: Implementation Complete", but the document still reads like a forward-looking migration plan (phases, future-tense tasks, milestones, rollback strategy). This is internally inconsistent and can mislead readers—either update the status to reflect that this is a plan/living doc, or revise sections (timeline/tasks wording) to reflect a completed migration with outcomes.

[Copilot]

The document hard-codes a branch name ("Switch-to-std-coroutines") in the header. Since branch names can change and the PR metadata indicates a different spelling, consider removing the branch field or ensuring it matches the actual branch name used for this work to avoid stale/incorrect documentation.

[Copilot]

This task list still describes the original planned PathFindAwaiter/co_await-based RipplePathFind migration, but the plan document/PR description notes the implementation diverged (condition_variable blocking wait, Context::coro removed, etc.). To keep this checklist useful, update the affected tasks to match the implemented approach (or clearly label this file as the pre-implementation plan checklist).

[Copilot]

The cspell words: list appears roughly alphabetized, but the new entry Mankawde is inserted after MEMORYSTATUSEX, which breaks ordering and can increase future merge conflicts. Please place the new word in the appropriate sorted position (or follow whatever ordering rule this list is intended to use).

Suggested change

	- MEMORYSTATUSEX
	- Mankawde
	- Mankawde
	- MEMORYSTATUSEX

[github-actions]

This PR has conflicts, please resolve them in order for the PR to be reviewed.

[github-actions]

All conflicts have been resolved. Assigned reviewers can now start or resume their review.

[xrplf-ai-reviewer]

No issues.

Review by Claude Opus 4.6 · Prompt: V15