Skip to content

Fix potential auto restart failure due to blocksync always true#2892

Closed
yzang2019 wants to merge 2 commits intomainfrom
yzang/fix-autoremediation
Closed

Fix potential auto restart failure due to blocksync always true#2892
yzang2019 wants to merge 2 commits intomainfrom
yzang/fix-autoremediation

Conversation

@yzang2019
Copy link
Copy Markdown
Contributor

Describe your changes and provide context

Problem identified:

  • In autoRestartIfBehind we use the blockSync as a flag to determine if already lag behind to avoid additional restart signals.
  • At the same time, the restart has a "cool down" period -- meaning, restart signals during cool down period are silently ignored.
  • Now, if a node restarts during cool down period, the signal gets ignored and the blockSync flag will remain indefinitely true effectively disabling the self-remediation process

Fix:

  • Do not set blocksync mode in auto restart handler

Testing performed to validate your change

Added unit test

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 13, 2026

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedFeb 13, 2026, 7:55 PM

@codecov
Copy link
Copy Markdown

codecov Bot commented Feb 13, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 48.35%. Comparing base (7407960) to head (3aad0c6).
⚠️ Report is 238 commits behind head on main.

❗ There is a different number of reports uploaded between BASE (7407960) and HEAD (3aad0c6). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (7407960) HEAD (3aad0c6)
sei-chain 1 0
Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##             main    #2892       +/-   ##
===========================================
- Coverage   57.15%   48.35%    -8.81%     
===========================================
  Files        2091      671     -1420     
  Lines      171524    50615   -120909     
===========================================
- Hits        98033    24474    -73559     
+ Misses      64751    24000    -40751     
+ Partials     8740     2141     -6599     
Flag Coverage Δ
sei-chain ?
sei-cosmos 48.16% <ø> (+<0.01%) ⬆️
sei-db 68.72% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 1543 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.


// Reset cooldown timer before sending to prevent rapid-fire signals
// regardless of whether the app-level restart accepts or rejects this signal
r.lastRestartTime = time.Now()

Check warning

Code scanning / CodeQL

Calling the system time Warning

Calling the system time may be a possible source of non-determinism
@github-actions
Copy link
Copy Markdown

We truly appreciate your contribution and the time you’ve invested in this PR.
Before we can merge it, we’d love your help addressing the remaining feedback or sharing your perspective.
If we don’t hear back within 2 day(s), this PR will close automatically — but don’t worry, you can reopen it anytime when you’re ready to continue.

@masih
Copy link
Copy Markdown
Collaborator

masih commented Apr 10, 2026

@yzang2019 OK to close this?

@yzang2019 yzang2019 closed this Apr 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants