[Limiter] Add RuleBook persistent data model and writer/diff logic by tillrohrmann · Pull Request #4677 · restatedev/restate

tillrohrmann · 2026-04-30T13:49:04Z

Introduces the cluster-global rule book that backs the in-memory Rules store of each partition processor's UserLimiter. This commit lands the foundational data layer for issue #4655:

RuleBook / PersistedRule / PersistedUserLimits in restate-limiter::rule_book, bilrost-encoded, Versioned. The RuleBook is keyed by RuleId (xxh3-64 of the rule pattern's canonical display form) rendered as rul_… resource ids. A deliberate 64-bit hash is used instead of the 128-bit norm for rendered-id brevity; the doc comment captures the trade-off.
Soft-tombstone semantics: PersistedRule.disabled: bool defaults to false so an active rule is bilrost's empty state and gets omitted from the wire.
Writer logic: RuleBook::apply_change for Create / Patch / Delete, with the version-bump contract — create/recreate uses the new book version, runtime-relevant patches advance the per-rule version, reason-only edits bump only last_modified and the book version, no-ops bump nothing. Hard cap on total rules (MAX_RULES_PER_BOOK) — configurable knob comes later.
RuleBook::diff: presence + per-rule version drives Vec<RuleUpdate> for the runtime, with disabled rules treated as absent. diff_from_empty for bootstrap consumers.

Supporting infra:

Adds Rule("rul") to IdResourceType; promotes restate-types::id_util and base62_util to public modules and IdEncoder::{new,push_u64,push_u128} to public so external crates can implement ResourceId.
Moves UserLimits and RuleUpdate from restate-worker-api into restate-limiter
Adds generic-array to the workspace deps.

claude

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

tillrohrmann · 2026-04-30T15:13:46Z

+/// here as a constant for now — Step 6 makes this configurable via
+/// `worker.rule_book.max_rules`.


tillrohrmann · 2026-04-30T15:14:08Z

@codex review

tillrohrmann · 2026-04-30T15:15:27Z

+/// Collisions are detectable at admin write time and surface as a 409 from
+/// the create endpoint.


But nothing can be done about them if they occur.

chatgpt-codex-connector · 2026-04-30T15:21:57Z

Codex Review: Didn't find any major issues. 🚀

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

tillrohrmann · 2026-04-30T15:22:37Z

+# Enables the persistent rule book (RuleBook, PersistedRule, RuleId,
+# diff/apply_change machinery). Implies `bilrost`.
+rule-book = [
+    "bilrost",
+    "dep:bytes",
+    "dep:generic-array",
+    "dep:restate-clock",
+    "dep:restate-encoding",
+    "dep:serde",
+    "dep:xxhash-rust",
+]


This feature is probably an overkill and we should either remove the features or move the rule-book to a different crate. Will become clearer as the user facing API for the rule book materializes.

tillrohrmann · 2026-04-30T15:23:29Z

    }
    /// Appends a u64 value as a padded base62 encoded string to the underlying buffer
-    pub(crate) fn push_u64(&mut self, i: u64) {
+    pub fn push_u64(&mut self, i: u64) {


Used to implement ResourceIds outside of the restate-types crate.

AhmedSoliman · 2026-05-05T09:42:40Z

+    pub disabled: bool,
+    /// Last time this rule was modified
+    #[bilrost(tag(6))]
+    pub last_modified: RoughTimestamp,


What's the reasoning behind choosing RoughTimestamp here?

Primarily to safe a few bytes as I thought that a resolution of seconds for when the rule was last modified is fine enough for all practical purposes.

AhmedSoliman · 2026-05-05T09:59:01Z

+    #[bilrost(tag(1))]
+    version: Version,
+    #[bilrost(tag(2), encoding(map<general, general>))]
+    rules: HashMap<RuleId, PersistedRule>,


One option is to use the string representation of the rule pattern as the key directly and remove it from the value. We can leave it as as a raw ReString or RulePattern` if it's strictly necessary.

The idea for introducing the RuleId was to have a handle for the REST API to refer to rules when patching them for example. The stringified version of the pattern would also work but it will result in slightly less nice looking URLs:

curl -X PATCH localhost:9070/rules/rul_XYZ vs. curl -X PATCH localhost:9070/rules/*/foobar/*

Alternatively, we can send the pattern as part of the request body if specifying it as part of the path is not so nice.

Another reason was that @slinkydeveloper has concerns about exposing the RulePattern to users as the wildcards can be confusing. We had a discussion whether a higher level language would be better to express the rule patterns https://www.notion.so/restatedev/Concurrency-limits-API-31ebfec2629180e3b091cb72e506b86d?source=copy_link#321bfec2629180719a0fc7f099b22f2d. So far we didn't find a good alternative, though.

The character * cannot be used in an url, needs to be percent encoded. I suggest the rule goes in the body

Introduces the cluster-global rule book that backs the in-memory `Rules` store of each partition processor's `UserLimiter`. This commit lands the foundational data layer for issue restatedev#4655 (steps 1 and 2 of the plan): * `RuleBook` / `PersistedRule` / `PersistedUserLimits` in `restate-limiter::rule_book`, bilrost-encoded, `Versioned`. The `RuleBook` is keyed by `RuleId` (xxh3-64 of the rule pattern's canonical display form) rendered as `rul_…` resource ids. A deliberate 64-bit hash is used instead of the 128-bit norm for rendered-id brevity; the doc comment captures the trade-off. * Soft-tombstone semantics: `PersistedRule.disabled: bool` defaults to `false` so an active rule is bilrost's empty state and gets omitted from the wire. * Writer logic: `RuleBook::apply_change` for `Create` / `Patch` / `Delete`, with the version-bump contract — create/recreate uses the new book version, runtime-relevant patches advance the per-rule version, reason-only edits bump only `last_modified` and the book version, no-ops bump nothing. Hard cap on total rules (`MAX_RULES_PER_BOOK`) — configurable knob comes later. * `RuleBook::diff`: presence + per-rule version drives `Vec<RuleUpdate>` for the runtime, with `disabled` rules treated as absent. `diff_from_empty` for bootstrap consumers. Supporting infra: * Adds `Rule("rul")` to `IdResourceType`; promotes `restate-types::id_util` and `base62_util` to public modules and `IdEncoder::{new,push_u64,push_u128}` to public so external crates can implement `ResourceId`. * Moves `UserLimits` and `RuleUpdate` from `restate-worker-api` into `restate-limiter` * Adds `generic-array` to the workspace deps. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…ogic

claude Bot reviewed Apr 30, 2026

View reviewed changes

tillrohrmann force-pushed the issues/4655 branch 2 times, most recently from 9af151a to 4d8d0ad Compare April 30, 2026 15:12

tillrohrmann commented Apr 30, 2026

View reviewed changes

tillrohrmann force-pushed the issues/4655 branch from 4d8d0ad to cf8c83b Compare April 30, 2026 15:28

tillrohrmann requested a review from AhmedSoliman April 30, 2026 15:28

tillrohrmann force-pushed the issues/4655 branch from cf8c83b to b713d9b Compare May 5, 2026 08:10

tillrohrmann mentioned this pull request May 5, 2026

[Limiter] Persist rule book per partition via UpsertRuleBook log entry #4689

Open

AhmedSoliman reviewed May 5, 2026

View reviewed changes

tillrohrmann force-pushed the issues/4655 branch from 0ea7af2 to 7eca992 Compare May 5, 2026 12:51

tillrohrmann and others added 2 commits May 5, 2026 16:22

fixup! [Limiter] Add RuleBook persistent data model and writer/diff l…

0c53a02

…ogic

tillrohrmann force-pushed the issues/4655 branch from 7eca992 to 0c53a02 Compare May 5, 2026 14:22

tillrohrmann mentioned this pull request May 5, 2026

[Limiter] Add admin rule book endpoints and push to local cache #4691

Open

		/// here as a constant for now — Step 6 makes this configurable via
		/// `worker.rule_book.max_rules`.

		/// Collisions are detectable at admin write time and surface as a 409 from
		/// the create endpoint.

Conversation

tillrohrmann commented Apr 30, 2026

Uh oh!

claude Bot left a comment

Choose a reason for hiding this comment

Claude Code Review

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tillrohrmann commented Apr 30, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot commented Apr 30, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

slinkydeveloper May 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

slinkydeveloper May 5, 2026 •

edited

Loading