feat: Implement Least Request Load Balancing Policy (gRFC A48) by emil10001 · Pull Request #2651 · grpc/grpc-rust

emil10001 · 2026-05-21T22:17:34Z

Implements the "all weights equal" Least Request Load Balancing policy in gRPC-Rust, in compliance with gRFC A48. The Least Request policy improves tail latencies in heterogeneous environments by tracking active request counts per endpoint and directing new requests to the backend with the lowest load.

Detailed Changes:

Core Load Balancing Policy (least_request.rs):
- Defined LeastRequestLoadBalancingConfig to parse and validate the choiceCount config parameter (default = 2, clamped from 2 to 10).
- Implemented LeastRequestBuilder registering policy name least_request_experimental.
- Implemented LeastRequestPolicy managing endpoint-level connections via ChildManager children delegating to pick_first.
- Maintained a persistent mapping of weak subchannel references to active request counters (subchannel_counters) so that outstanding request metrics survive picker updates and name re-resolutions.
- Implemented LeastRequestPicker utilizing a random sampling selection algorithm over choice_count subchannels.
Active Request Cancellation Safety:
- Identified and resolved a request counter leak bug where async task cancellations during dyn_invoke.await dropped the Pick closure without calling it.
- Implemented a custom, defusable ActiveRequestGuard using an AtomicBool inside LeastRequestPicker::pick. The guard guarantees that the active request count is decremented upon drop if the picker's on_complete callback is never invoked.
Channel & Service Config Integration:
- Registered the builder with the global LB registry in Channel::new inside channel.rs.
- Added CallbackRecvStream wrapping the stream in the channel's Invoke implementation to trigger on_complete callbacks when client streams are completed or dropped.
- Added LeastRequest variant to LbPolicyType enum in service_config.rs.
- Mapped LbPolicyType::LeastRequest configuration inside ResolverChannelController::update in channel.rs.
Test Additions & Verification:
- Added comprehensive unit tests in least_request.rs covering configuration parsing/clamping/validation, least request selection, tie-breaking, fewer subchannels than choice count, and cancellation drop-guard safety.
- Modified the InMemoryResolver in inmemory/mod.rs to dynamically set the LeastRequest load-balancing policy based on target URI path prefixes.
- Wrote a robust E2E integration test test_in_memory_least_request_load_balancing in inmemory/mod.rs verifying dynamic load balancing across multiple in-memory backends concurrently.

Motivation

Solution

Implements the "all weights equal" Least Request Load Balancing policy in gRPC-Rust, in compliance with gRFC A48. The Least Request policy improves tail latencies in heterogeneous environments by tracking active request counts per endpoint and directing new requests to the backend with the lowest load. Detailed Changes: 1. Core Load Balancing Policy (`least_request.rs`): - Defined `LeastRequestLoadBalancingConfig` to parse and validate the `choiceCount` config parameter (default = 2, clamped from 2 to 10). - Implemented `LeastRequestBuilder` registering policy name `least_request_experimental`. - Implemented `LeastRequestPolicy` managing endpoint-level connections via `ChildManager` children delegating to `pick_first`. - Maintained a persistent mapping of weak subchannel references to active request counters (`subchannel_counters`) so that outstanding request metrics survive picker updates and name re-resolutions. - Implemented `LeastRequestPicker` utilizing a random sampling selection algorithm over `choice_count` subchannels. 2. Active Request Cancellation Safety: - Identified and resolved a request counter leak bug where async task cancellations during `dyn_invoke.await` dropped the `Pick` closure without calling it. - Implemented a custom, defusable `ActiveRequestGuard` using an `AtomicBool` inside `LeastRequestPicker::pick`. The guard guarantees that the active request count is decremented upon drop if the picker's `on_complete` callback is never invoked. 3. Channel & Service Config Integration: - Registered the builder with the global LB registry in `Channel::new` inside `channel.rs`. - Added `CallbackRecvStream` wrapping the stream in the channel's `Invoke` implementation to trigger `on_complete` callbacks when client streams are completed or dropped. - Added `LeastRequest` variant to `LbPolicyType` enum in `service_config.rs`. - Mapped `LbPolicyType::LeastRequest` configuration inside `ResolverChannelController::update` in `channel.rs`. 4. Test Additions & Verification: - Added comprehensive unit tests in `least_request.rs` covering configuration parsing/clamping/validation, least request selection, tie-breaking, fewer subchannels than choice count, and cancellation drop-guard safety. - Modified the `InMemoryResolver` in `inmemory/mod.rs` to dynamically set the `LeastRequest` load-balancing policy based on target URI path prefixes. - Wrote a robust E2E integration test `test_in_memory_least_request_load_balancing` in `inmemory/mod.rs` verifying dynamic load balancing across multiple in-memory backends concurrently.

emil10001 · 2026-05-22T17:47:43Z

/gemini review

dfawley

Thanks for the PR! I haven't looked at the tests yet, but here's a first round of review comments.

dfawley · 2026-05-22T00:14:56Z

    #[default]
    PickFirst,
    RoundRobin,
+    LeastRequest,


This is a pre-defined list of LB policies limited to just PF and RR. But I think we can do this for now if we put a TODO to remove. cc @nathanielford for when he implements support for the newer field that lets you specify any policy.

dfawley · 2026-05-22T00:16:30Z

    fn build(&self, target: &Target, options: ResolverOptions) -> Box<dyn Resolver> {
        let path = target.path().strip_prefix('/').unwrap_or(target.path());
-        let ids: Vec<String> = path.split(',').map(|s| s.to_string()).collect();
+        let (lb_policy, rest) = if let Some(stripped) = path.strip_prefix("leastrequest/") {


The LB policy should get set by specifying a default service config, not through the in memory resolver. We don't have that yet, though, so this is fine for now since this module is unexported and unused except in tests.

dfawley · 2026-05-22T00:22:57Z

+    fn parse_config(
+        &self,
+        config: &ParsedJsonLbConfig,
+    ) -> Result<Option<<Self::LbPolicy as LbPolicy>::LbConfig>, String> {


I believe you can just name the type here if you want to avoid all the colons and "as"es.. Result<LeastRequestLoadBalancingConfig, String>?

dfawley · 2026-05-22T00:25:15Z

+        let choice_count = parsed.choice_count.min(10);
+        Ok(Some(LeastRequestLoadBalancingConfig { choice_count }))


Optional: A style choice. This could also be parsed.choice_count = ...min(10) and then return Ok(Some(parsed)). (parsed would need to be declared as mut)

dfawley · 2026-05-22T00:28:37Z

+pub(crate) struct LeastRequestPolicy {
+    child_manager: ChildManager<Endpoint>,
+    pick_first_builder: Arc<DynLbPolicyBuilder>,
+    choice_count: u32,


Consider storing an Option<Config> here instead of the flat config fields.

dfawley · 2026-05-22T22:47:49Z

+        PickResult::Pick(crate::client::load_balancing::Pick {
+            subchannel: selected.subchannel.clone(),
+            metadata: crate::metadata::MetadataMap::new(),


use please.

dfawley · 2026-05-22T22:59:08Z

+                        .state
+                        .picker
+                        .pick(&crate::core::RequestHeaders::default())


We should not be making picks for fake RPCs. If we can't map via the child itself, then we'll need another way to retrieve its active subchannel.

dfawley · 2026-05-22T23:00:01Z

+            // Clean up stale counters
+            self.subchannel_counters
+                .retain(|weak, _| weak.upgrade().is_some());


I think this would be better if it reused the known-active subchannels that we got above instead of keeping all the ones that happen to still upgrade.

dfawley · 2026-05-22T23:05:35Z

+        }
+        let aggregate_state = self.child_manager.aggregate_states();
+
+        if aggregate_state == ConnectivityState::Ready {


Optional: maybe let picker = if ... and send the update commonly between the if/else.

dfawley · 2026-05-22T23:07:35Z

+            let picker = self
+                .child_manager
+                .children()
+                .find(|cs| cs.state.connectivity_state == aggregate_state)
+                .map(|cs| cs.state.picker.clone())
+                .unwrap_or_else(|| {
+                    Arc::new(crate::client::load_balancing::QueuingPicker) as Arc<dyn Picker>
+                });


What picker is coming out of this? Is it just the first child whose state is the same as the aggregate state? We should round-robin over all of those children that match instead. You should ideally share the existing RoundRobin picker for this.

emil10001 force-pushed the master branch from 24b4a7d to 87b8680 Compare May 21, 2026 22:50

emil10001 added 2 commits May 21, 2026 15:52

Merge branch 'grpc:master' into master

d4737a7

Merge branch 'master' into master

70d540d

dfawley reviewed May 22, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Implement Least Request Load Balancing Policy (gRFC A48)#2651

feat: Implement Least Request Load Balancing Policy (gRFC A48)#2651
emil10001 wants to merge 3 commits into
grpc:masterfrom
emil10001:master

emil10001 commented May 21, 2026

Uh oh!

emil10001 commented May 22, 2026

Uh oh!

dfawley left a comment

Uh oh!

dfawley May 22, 2026

Uh oh!

dfawley May 22, 2026

Uh oh!

dfawley May 22, 2026

Uh oh!

dfawley May 22, 2026

Uh oh!

dfawley May 22, 2026

Uh oh!

dfawley May 22, 2026

Uh oh!

dfawley May 22, 2026

Uh oh!

dfawley May 22, 2026

Uh oh!

dfawley May 22, 2026

Uh oh!

dfawley May 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		let choice_count = parsed.choice_count.min(10);
		Ok(Some(LeastRequestLoadBalancingConfig { choice_count }))

Conversation

emil10001 commented May 21, 2026

Motivation

Solution

Uh oh!

emil10001 commented May 22, 2026

Uh oh!

dfawley left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants