Add multi-worker serve mode for Sendspin Party#199
Draft
maximmaxim345 wants to merge 32 commits intomainfrom
Draft
Add multi-worker serve mode for Sendspin Party#199maximmaxim345 wants to merge 32 commits intomainfrom
maximmaxim345 wants to merge 32 commits intomainfrom
Conversation
asyncio.wait_for + run_in_executor with a blocking queue.get() leaves orphaned threads that consume messages and discard them when the future is cancelled. Use queue.get(timeout=0.5) instead so the thread exits cleanly on timeout.
The web player on a worker port (e.g. :9002) fetches the coordinator's /api/status (e.g. :9000) which is cross-origin. Add Access-Control-Allow-Origin: * to allow the request.
Cancel the running task on SIGINT so it breaks out of blocking audio decode. Ignore repeated Ctrl+C after the first one.
When the last client disconnects from a worker, the stream is stopped. Catch StreamStoppedError and clear the stream reference so the worker skips audio chunks until a new client connects and creates a fresh stream.
- Add worker crash detection: coordinator checks process liveness every 30s and removes dead workers from the redirect pool. Shuts down if all workers crash. - Validate --workers >= 1 in CLI - Add CORS header to single-worker /api/status for consistency - Use _queue.Empty instead of bare Exception in _drain_status_queue - Remove unused TYPE_CHECKING import and empty pass block - Redirect handler now uses live _active_worker_ports (reflects crash removal) - Frontend: hide listener count when stopped, show min 1 when listening
Track already-reported crashes so _check_worker_health doesn't print the same crash message every 30s for workers that are already dead.
Clear _active_group alongside _stream on StreamStoppedError so that the next connecting client creates a fresh group and stream instead of being added to a stale group with no active stream.
The coordinator was pacing itself to real-time (sleeping when ahead > 0), resulting in only ~250ms of buffer on workers. Allow up to 5s of buffer ahead, matching single-worker mode's max_buffer_us=5_000_000.
Workers now use consecutive ports starting at --port instead of --port+1.
Prevents reconnect race where a client joins a dead group whose stream was already stopped by aiosendspin.
Multi-worker mode does not support outbound client connections. Previously the --client flag was silently ignored.
Track successful vs failed workers separately. Exit if all workers fail. Remove dead audio queues for failed workers.
Previously only shut down when all workers crashed.
Coordinator now loops between waiting for clients and streaming, instead of continuously decoding when nobody is listening.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds multi-worker support to
sendspin serveso multiple worker processes can accept Sendspin Party listeners while sharing thesame decoded audio timeline.
It introduces a coordinator/worker architecture where:
Changes
--workerstosendspin serve/api/statusendpoint for total listener count--clientwhen used with multi-worker modeREADME.mdNotes
--clientremains supported in single-worker serve mode only