Skip to content

fix(streams): release AR connections after each dispatcher iteration#158

Merged
mhenrixon merged 1 commit into
mainfrom
fix/dispatcher-ar-connection-leak
May 20, 2026
Merged

fix(streams): release AR connections after each dispatcher iteration#158
mhenrixon merged 1 commit into
mainfrom
fix/dispatcher-ar-connection-leak

Conversation

@mhenrixon
Copy link
Copy Markdown
Owner

@mhenrixon mhenrixon commented May 20, 2026

Summary

Fixes the AR connection leak in StreamEventDispatcher that causes 10s request wedges in development on every Rails code reload when the dashboard/SSE is open.

The dispatcher runs in a background fiber (Thread.new { run_loop }). When StreamStat.record! fires, it leases an AR connection from the BusRecord pool. After the INSERT, the fiber parks on @queue.pop — but the connection stays leased to the parked fiber.

Under config.active_support.isolation_level = :fiber (common in modern Rails apps), AR connection leases are keyed to the owning Fiber. The dispatcher's fiber never dies and never calls clear_active_connections!, so the lease persists for the life of the worker.

On the next dev file save, the Rails reloader runs clear_reloadable_connections!, which calls attempt_to_checkout_all_existing_connections on every pool. The :pgbus pool has exactly one connection leased to the parked dispatcher fiber — so it blocks until rack-timeout fires (10s).

Fix: call BusRecord.connection_handler.clear_active_connections! in an ensure block at the end of each run_loop iteration. This releases any AR connections the dispatcher acquired during message handling before parking on the queue.

begin
  # handle message(s)
ensure
  release_ar_connections
end

The ensure guarantees cleanup even when handle() raises. The method is guarded by defined?(::ActiveRecord::Base) so non-Rails contexts aren't affected, and errors in the release itself are swallowed at debug level.

Impact in prod: beyond fixing the dev wedge, this also reclaims the wasted connection-per-puma-worker that was permanently leased in production. One fewer idle connection per worker on the :pgbus pool.

Test plan

  • bundle exec rspec spec/pgbus/web/streamer/ — 160 examples, 0 failures
  • New specs:
    • "releases ActiveRecord connections after processing a message" — verifies clear_active_connections! is called after a normal message
    • "releases connections even when handle raises" — verifies the ensure path
  • bundle exec rubocop — clean
  • Full unit suite: 2017 examples, 0 failures
  • Manual: bin/dev in a Zazu-like app → open dashboard (subscribes to SSE) → save files rapidly → no 10s wedge

Summary by CodeRabbit

  • Bug Fixes

    • Enhanced event streaming reliability by improving database connection management. The system now consistently releases connections after processing each message, preventing potential resource exhaustion and improving overall system stability.
  • Tests

    • Added test coverage for database connection release behavior, including error scenarios.

Review Change Stack

The StreamEventDispatcher runs in a background fiber and acquires an
ActiveRecord connection via StreamStat.record! (BusRecord.create!).
After the INSERT, the fiber parks on @queue.pop — but the connection
stays leased to the parked fiber. On the next Rails code reload,
clear_reloadable_connections! blocks waiting for the leased connection,
wedging the request thread for the full rack-timeout duration (10s).

Under config.active_support.isolation_level = :fiber, AR connection
leases are keyed to the owning Fiber. The dispatcher's fiber never
dies and never calls clear_active_connections!, so the lease persists
for the life of the worker.

Fix: call BusRecord.connection_handler.clear_active_connections! in
an ensure block at the end of each run_loop iteration. This releases
any AR connections the dispatcher acquired during message handling
before parking on the queue. The ensure guarantees cleanup even when
handle() raises.

Reported by Juraj (getzazu/app#2310).
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 20, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI (base), Organization UI (inherited)

Review profile: ASSERTIVE

Plan: Pro

Run ID: b2c04315-5df1-411b-a316-d7833aa56875

📥 Commits

Reviewing files that changed from the base of the PR and between 303e7f9 and 6198998.

📒 Files selected for processing (2)
  • lib/pgbus/web/streamer/stream_event_dispatcher.rb
  • spec/pgbus/web/streamer/stream_event_dispatcher_spec.rb

📝 Walkthrough

Walkthrough

Stream event dispatcher now ensures ActiveRecord connections are released after each event loop iteration by adding a new release_ar_connections method and wrapping message handling in an ensure block that invokes it unconditionally, with tests verifying the cleanup occurs both on success and exception.

Changes

Connection Release Mechanism

Layer / File(s) Summary
Connection release implementation and integration
lib/pgbus/web/streamer/stream_event_dispatcher.rb
New release_ar_connections method clears active ActiveRecord connections via ConnectionHandler.clear_active_connections! with exception rescue and debug logging, integrated into run_loop via an ensure block that guarantees execution after each message-processing iteration.
Connection release test coverage
spec/pgbus/web/streamer/stream_event_dispatcher_spec.rb
Test suite verifies clear_active_connections! is called at least once per loop iteration in both normal message handling and exception scenarios (when read_after raises).

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Suggested labels

bug, streaming

Poem

🐰 Connections pooled and waiting,
Each message comes, ActiveRecord's play,
An ensure block, gently breaking free,
No lingering links, no leaks to stay—
The event stream hops away clean! 🐇

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'fix(streams): release AR connections after each dispatcher iteration' directly and clearly summarizes the main change: releasing ActiveRecord connections in the stream dispatcher after each iteration.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/dispatcher-ar-connection-leak

Comment @coderabbitai help to get the list of available commands and usage tips.

@mhenrixon mhenrixon self-assigned this May 20, 2026
@mhenrixon mhenrixon merged commit 60fc968 into main May 20, 2026
8 of 9 checks passed
@mhenrixon mhenrixon deleted the fix/dispatcher-ar-connection-leak branch May 20, 2026 12:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant