Skip to content

Add MaxGauge metric#464

Open
gabotechs wants to merge 1 commit into
gabrielmusat/network-boundary-builderfrom
gabrielmusat/max-gauge
Open

Add MaxGauge metric#464
gabotechs wants to merge 1 commit into
gabrielmusat/network-boundary-builderfrom
gabrielmusat/max-gauge

Conversation

@gabotechs
Copy link
Copy Markdown
Collaborator

@gabotechs gabotechs commented May 24, 2026

This is one PR from the following stack of PRs:

This PR introduces a MaxGauge metric to provide better tracking of peak values in distributed metrics collection. This enables more accurate monitoring of resource utilization and helps identify bottlenecks in the execution pipeline.

@gabotechs gabotechs changed the title Gabrielmusat/max gauge Add MaxGauge metric May 24, 2026
@gabotechs gabotechs force-pushed the gabrielmusat/max-gauge branch 5 times, most recently from 8b79520 to 7c61fdb Compare May 25, 2026 08:54
@gabotechs gabotechs changed the base branch from main to gabrielmusat/network-boundary-builder May 25, 2026 08:55
@gabotechs gabotechs force-pushed the gabrielmusat/max-gauge branch from 7c61fdb to 3380075 Compare May 25, 2026 18:55
@gabotechs gabotechs force-pushed the gabrielmusat/network-boundary-builder branch from 382dc95 to 110bbce Compare May 26, 2026 13:22
@gabotechs gabotechs force-pushed the gabrielmusat/max-gauge branch from 3380075 to c471253 Compare May 26, 2026 13:22
@gabotechs gabotechs force-pushed the gabrielmusat/network-boundary-builder branch from 110bbce to 567c155 Compare May 26, 2026 13:53
@gabotechs gabotechs force-pushed the gabrielmusat/max-gauge branch from c471253 to d7111ff Compare May 26, 2026 13:53
gabotechs added a commit that referenced this pull request May 26, 2026
… be the same (#427)

This is one PR from the following stack of PRs:
- #427
<- you are here
- #469
- #461
- #462
- #463
- #464
- #432

This is a preparatory step towards:
-
#377

This is an optimization that allows workers to communicate in-memory
avoiding network calls and serialization in case it needs to communicate
to itself.

This optimization shows good improvements if using a small number of
workers, and fades away as more workers are used.

Even if this shows improvements today, it will become very meaningful in
adaptative query execution: there will be times when two consecutive
stages are assigned 1 task each, and as that was done dynamically, we
cannot eagerly do the optimization that collapses those two stages into
one, so instead, we assigned both stages to the same worker, and the
optimization in this PR kicks in, executing the plan fully locally.
@gabotechs gabotechs force-pushed the gabrielmusat/network-boundary-builder branch from 567c155 to d1c7428 Compare May 26, 2026 18:05
@gabotechs gabotechs force-pushed the gabrielmusat/max-gauge branch from d7111ff to 4260f55 Compare May 26, 2026 18:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant