Skip to content

feat(proto): expose max_transmit_segments and max_transmit_datagrams in TransportConfig#645

Open
poka-IT wants to merge 1 commit into
n0-computer:mainfrom
poka-IT:warren-perf-tier1-setter-api
Open

feat(proto): expose max_transmit_segments and max_transmit_datagrams in TransportConfig#645
poka-IT wants to merge 1 commit into
n0-computer:mainfrom
poka-IT:warren-perf-tier1-setter-api

Conversation

@poka-IT
Copy link
Copy Markdown

@poka-IT poka-IT commented May 8, 2026

Description

Replace the hardcoded constants MAX_TRANSMIT_SEGMENTS = 10 and MAX_TRANSMIT_DATAGRAMS = 20 in noq::Connection::drive_transmit with two configurable fields on TransportConfig. Defaults are unchanged (10 / 20), so default behavior is preserved.

New API:

let mut t = TransportConfig::default();
t.max_transmit_segments(NonZeroUsize::new(40).unwrap());
t.max_transmit_datagrams(NonZeroUsize::new(80).unwrap());

This is an alternative to #636 (which simply bumps the constants to 40 / 80 globally). After feedback from @flub there, exposing the values as a runtime knob seems more defensible:

  • No default-behavior change: multi-connection workloads keep their per-connection memory footprint.
  • Single-connection bulk-transfer workloads opt in: applications such as ours (peer-to-peer VPN tunnel over QUIC) can call transport.max_transmit_segments(NonZeroUsize::new(40).unwrap()) and get the throughput / CPU-efficiency gains demonstrated in the perf(connection): raise MAX_TRANSMIT_SEGMENTS to 40 and MAX_TRANSMIT_DATAGRAMS to 80 #636 benches.
  • Pattern-consistent: same shape as enable_segmentation_offload, congestion_controller_factory, etc.

Why a setter rather than a higher default

MAX_TRANSMIT_SEGMENTS = 10 is exactly equal to MIN_BURST_SIZE in noq-proto/src/connection/pacing.rs:162, the historical 10 was likely chosen to match the pacer's minimum burst. The pacer itself, however, already allows up to MAX_BURST_SIZE = 256 per burst, so even a setter value of 256 stays within the existing pacer envelope. CC and pacing remain authoritative; the constants are just an outer cap.

Trade-offs now self-documenting

Breaking Changes

n/a, defaults preserved (10 / 20).

Notes & open questions

Bench data motivating both PRs (paired same-session, 3x CCX23 Hetzner, real network: intra-DC and inter-region EU) is in the discussion thread of #636.

Change checklist

  • Self-review.
  • Tests (defaults preserved + setter behavior).
  • No breaking changes.

@poka-IT poka-IT force-pushed the warren-perf-tier1-setter-api branch from f0f183c to ade217c Compare May 8, 2026 03:44
@n0bot n0bot Bot added this to iroh May 8, 2026
@github-project-automation github-project-automation Bot moved this to 🚑 Needs Triage in iroh May 8, 2026
@dignifiedquire
Copy link
Copy Markdown
Contributor

I like this direction, as it reflects reality better, that these values need to be different depending on actual usage.

Copy link
Copy Markdown
Collaborator

@flub flub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I also think this is a good direction. Doesn't mean the other PR shouldn't happen, but let's do it on top of this one.

Comment thread noq-proto/src/config/transport.rs Outdated
Comment thread noq-proto/src/connection/mod.rs Outdated
Comment thread noq-proto/src/tests/mod.rs Outdated
Comment thread noq-proto/src/config/transport.rs Outdated
Comment thread noq-proto/src/tests/mod.rs
@github-project-automation github-project-automation Bot moved this from 🚑 Needs Triage to 🏗 In progress in iroh May 11, 2026
@poka-IT poka-IT force-pushed the warren-perf-tier1-setter-api branch from ade217c to e0b77c4 Compare May 11, 2026 18:59
@poka-IT
Copy link
Copy Markdown
Author

poka-IT commented May 11, 2026

Thanks for the thorough review. All five points addressed in e0b77c47:

1. Newline between the two fields removed (grouped with enable_segmentation_offload).

2. Both accessors on Connection deleted. To keep the change minimal:

  • max_transmit_segments is now clamped inside Connection::poll_transmit directly (via max_datagrams.min(self.config.max_transmit_segments)). The driver no longer has to read this value at all, the clamp follows the active config automatically. In a multi-transport future the clamp can be made per-path right where it lives.
  • max_transmit_datagrams is still needed by noq::Connection::drive_transmit (it gates the loop, not the inner poll_transmit call). It is now read once at construction in noq::endpoint (from the ClientConfig / ServerConfig available at that call site) and cached on noq::State, never queried again from Connection. For the case where accept is called without a per-call server_config and the endpoint default is not reachable from noq, I fall back on TransportConfig::default().get_max_transmit_datagrams() — happy to refine if you'd prefer threading the endpoint default through proto::Endpoint::accept's return.

Two small read-only helpers got added to keep the privacy story clean: ClientConfig::get_transport_config() and TransportConfig::get_max_transmit_datagrams() (same get_ prefix as the existing EndpointConfig::get_max_udp_payload_size).

3. default_max_transmit_settings test removed.

4. Defaults extracted as DEFAULT_MAX_TRANSMIT_SEGMENTS / DEFAULT_MAX_TRANSMIT_DATAGRAMS at the top of noq-proto/src/config/transport.rs, used in Default::default().

5. Setter test rewritten using stats-delta: handshake stats are sampled before sending the stream, then again after, and the assertion is datagrams_delta <= ios_delta * CAP. Deterministic — no averaging.

Side-effect: the rustdoc on both setters got compacted, and I trimmed the rustdoc on the enable_segmentation_offload neighbour to match.

@poka-IT
Copy link
Copy Markdown
Author

poka-IT commented May 11, 2026

This latest commit simply changes the default value based on the benchmark results outlined in PR #636.
It can be reverted depending on what you decide.

Comment thread noq-proto/src/config/transport.rs Outdated
Comment thread noq-proto/src/tests/mod.rs Outdated
Comment thread noq/src/connection.rs Outdated
Comment thread noq/src/connection.rs Outdated
Comment thread noq-proto/src/config/transport.rs Outdated
Comment thread noq-proto/src/config/transport.rs Outdated
Comment thread noq/src/connection.rs
@flub
Copy link
Copy Markdown
Collaborator

flub commented May 13, 2026

This latest commit simply changes the default value based on the benchmark results outlined in PR #636. It can be reverted depending on what you decide.

That's fine, we can do this in the same PR.

@poka-IT poka-IT force-pushed the warren-perf-tier1-setter-api branch from 5213ee8 to 411e21c Compare May 13, 2026 17:36
@poka-IT
Copy link
Copy Markdown
Author

poka-IT commented May 13, 2026

Thanks, all five addressed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: 🏗 In progress

Development

Successfully merging this pull request may close these issues.

3 participants