PromQL: use start timestamps for rate()-like calculations#18344
Merged
krajorama merged 21 commits intoApr 21, 2026
Merged
Conversation
Signed-off-by: Vilius Pranckaitis <vpranckaitis@gmail.com>
vpranckaitis
commented
Mar 23, 2026
krajorama
reviewed
Mar 24, 2026
Member
krajorama
left a comment
krajorama
left a comment
There was a problem hiding this comment.
some minor comments, this is looking pretty good
Signed-off-by: Vilius Pranckaitis <vpranckaitis@gmail.com>
Signed-off-by: Vilius Pranckaitis <vpranckaitis@gmail.com>
Signed-off-by: Vilius Pranckaitis <vpranckaitis@gmail.com>
vpranckaitis
changed the title
rate() and increase() calculationsrate()-like calculations
Signed-off-by: vpranckaitis <vpranckaitis@gmail.com>
vpranckaitis
force-pushed
the
use_start_timestamps_in_rate_like_functions
branch
from
28f1519 to
b5682a1
Compare
Signed-off-by: vpranckaitis <vpranckaitis@gmail.com>
krajorama
reviewed
Apr 10, 2026
Member
krajorama
left a comment
krajorama
left a comment
There was a problem hiding this comment.
Looks good, almost there. I've tried a number of times to wrap my head around the T=ST case ... seems ok, but maybe we could walk through it at the next WG meeting.
I've made a bunch of suggestions to the tests and code.
Signed-off-by: vpranckaitis <vpranckaitis@gmail.com>
Signed-off-by: vpranckaitis <vpranckaitis@gmail.com>
Signed-off-by: vpranckaitis <vpranckaitis@gmail.com>
Signed-off-by: vpranckaitis <vpranckaitis@gmail.com>
krajorama
reviewed
Apr 14, 2026
Signed-off-by: vpranckaitis <vpranckaitis@gmail.com>
krajorama
approved these changes
Apr 14, 2026
Member
krajorama
left a comment
krajorama
left a comment
There was a problem hiding this comment.
LGTM.
I think it's too early to add it to features API. We'll need native histograms support for that and extended range selectors.
Signed-off-by: vpranckaitis <vpranckaitis@gmail.com>
roidelapluie
approved these changes
Apr 16, 2026
vamsi-01
added a commit
to vamsi-01/prometheus
that referenced
this pull request
Apr 20, 2026
Implement detection of overlapping delta aggregation windows where a sample's start timestamp is less than the previous sample's timestamp. This resolves issue prometheus#18534 by: - Collecting start timestamps during matrix evaluation - Checking for overlaps in both float and histogram samples - Emitting DeltaStartTimeOverlapWarning annotations when detected - Warning once per series to avoid spam The implementation builds on PR prometheus#18344's start timestamp infrastructure, which propagates ST values through the query path via StartTimestamps struct. Changes: - Add DeltaStartTimeOverlapWarning annotation type with merging support - Implement checkDeltaStartTimeOverlaps() to detect overlapping windows - Enable ST collection in matrixSelector for overlap detection - Add comprehensive unit tests for overlap detection logic - Test various scenarios: normal case, overlaps, nil/zero STs Depends on: prometheus#18344 (PromQL: use start timestamps for rate-like calculations) Fixes: prometheus#18534 ```release-notes [FEATURE] PromQL: Annotate a warning when delta samples have overlapping start times (start time < previous timestamp). ``` Signed-off-by: Vamsi Mathala <vmathala@redhat.com>
Signed-off-by: vpranckaitis <vpranckaitis@gmail.com>
# Conflicts: # cmd/prometheus/main.go
Signed-off-by: vpranckaitis <vpranckaitis@gmail.com>
auto-merge was automatically disabled
April 21, 2026 14:53
Head branch was pushed to by a user without write access
vamsi-01
added a commit
to vamsi-01/prometheus
that referenced
this pull request
Apr 21, 2026
Implement detection of overlapping delta aggregation windows where a sample's start timestamp is less than the previous sample's timestamp. This resolves issue prometheus#18534 by: - Collecting start timestamps during matrix evaluation - Checking for overlaps in both float and histogram samples - Emitting DeltaStartTimeOverlapWarning annotations when detected - Warning once per series to avoid spam The implementation builds on PR prometheus#18344's start timestamp infrastructure, which propagates ST values through the query path via StartTimestamps struct. Changes: - Add DeltaStartTimeOverlapWarning annotation type with merging support - Implement checkDeltaStartTimeOverlaps() to detect overlapping windows - Enable ST collection in matrixSelector for overlap detection - Add comprehensive unit tests for overlap detection logic - Test various scenarios: normal case, overlaps, nil/zero STs Depends on: prometheus#18344 (PromQL: use start timestamps for rate-like calculations) Fixes: prometheus#18534 ```release-notes [FEATURE] PromQL: Annotate a warning when delta samples have overlapping start times (start time < previous timestamp). ``` Signed-off-by: Vamsi Mathala <vmathala@redhat.com>
vamsi-01
added a commit
to vamsi-01/prometheus
that referenced
this pull request
Apr 21, 2026
Implement detection of overlapping delta aggregation windows where a sample's start timestamp is less than the previous sample's timestamp. This resolves issue prometheus#18534 by: - Collecting start timestamps during matrix evaluation - Checking for overlaps in both float and histogram samples - Emitting DeltaStartTimeOverlapWarning annotations when detected - Warning once per series to avoid spam The implementation builds on PR prometheus#18344's start timestamp infrastructure, which propagates ST values through the query path via StartTimestamps struct. Changes: - Add DeltaStartTimeOverlapWarning annotation type with merging support - Implement checkDeltaStartTimeOverlaps() to detect overlapping windows - Enable ST collection in matrixSelector for overlap detection - Add comprehensive unit tests for overlap detection logic - Test various scenarios: normal case, overlaps, nil/zero STs Fixes: prometheus#18534 ```release-notes [FEATURE] PromQL: Annotate a warning when delta samples have overlapping start times (start time < previous timestamp). ``` Signed-off-by: Vamsi Mathala <vmathala@redhat.com>
vamsi-01
added a commit
to vamsi-01/prometheus
that referenced
this pull request
Apr 22, 2026
Implement detection of overlapping aggregation windows where a sample's start timestamp is less than the previous sample's timestamp. This applies to both delta and cumulative counter metrics. This resolves issue prometheus#18534 by: - Checking for overlaps in rate/increase/delta function calls - Using condition: currST != 0 && currST < prevT && currST != prevST - This correctly handles both delta and cumulative counter metrics - Emitting StartTimeOverlapWarning annotations when detected - Warning once per series to avoid spam - Only extracting metric name when overlap is detected (performance) The implementation builds on PR prometheus#18344's start timestamp infrastructure, which propagates ST values through the query path via StartTimestamps struct. Changes: - Add StartTimeOverlapWarning annotation type with merging support - Implement checkStartTimeOverlap() to detect overlapping windows - Add overlap checks in extrapolatedRate() for float samples - Add overlap checks in histogramRate() for histogram samples - Add promqltest tests for overlap detection scenarios - Remove engine.go matrixSelector check (moved to function call path) Fixes: prometheus#18534 Signed-off-by: Vamsi Mathala <vmathala@redhat.com>
vamsi-01
added a commit
to vamsi-01/prometheus
that referenced
this pull request
Apr 23, 2026
Implement detection of overlapping aggregation windows where a sample's start timestamp is less than the previous sample's timestamp. This applies to both delta and cumulative counter metrics. This resolves issue prometheus#18534 by: - Checking for overlaps in rate/increase/delta function calls - Using condition: currST != 0 && currST < prevT && currST != prevST - This correctly handles both delta and cumulative counter metrics - Emitting StartTimeOverlapWarning annotations when detected - Warning once per series to avoid spam - Only extracting metric name when overlap is detected (performance) The implementation builds on PR prometheus#18344's start timestamp infrastructure, which propagates ST values through the query path via StartTimestamps struct. Changes: - Add StartTimeOverlapWarning annotation type with merging support - Implement checkStartTimeOverlap() to detect overlapping windows - Add overlap checks in extrapolatedRate() for float samples - Add overlap checks in histogramRate() for histogram samples - Add promqltest tests for overlap detection scenarios - Remove engine.go matrixSelector check (moved to function call path) Fixes: prometheus#18534 Signed-off-by: Vamsi Mathala <vmathala@redhat.com>
rbizos
pushed a commit
to rbizos/prometheus
that referenced
this pull request
Apr 29, 2026
…us#18344) * PromQL: use start timestamps for rate() and increase() calculations * implement start timestamps reset detection for `irate()` * add `start_timestamps.test` * add a couple of tests with subqueries * add a test for cumulative with unknown start timestamp * update `enable-features` CLI parameter description * `make cli-documentation` Signed-off-by: vpranckaitis <vpranckaitis@gmail.com> --------- Signed-off-by: Vilius Pranckaitis <vpranckaitis@gmail.com> Signed-off-by: vpranckaitis <vpranckaitis@gmail.com> Signed-off-by: Raphael Bizos <r.bizos@criteo.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue(s) does the PR fix:
Implemented a change to use start timestamps for
rate()andincrease()calculations. This is a part of OTLP Delta Support project.After this change, counter resets are not only detected by looking at datapoint values, but also by checking start timestamps. As described in PROM-77 proposal, this allows querying deltas with
rate()andincrease()functions, as long as they have valid start timestamps. Additionally, it should also improve counter reset detection for cumulative counters, where the first scraped value after a counter reset is as high as it was before the reset.This PR also follows some of the ideas expressed in this comment by @enisoc which are meant to minimize memory usage impact when start timestamps usage is not enabled or when their values would be not useful for the PromQL evaluation.
This PR does not include the changes necessary for start timestamps to work when
anchoredandsmoothedmodifiers are used. It was agreed in OTLP Delta Support Working Group meetings that we want to first see a working end-to-end solution before addressing experimentalanchoredandsmoothedmodifiers.Does this PR introduce a user-facing change?