Skip to content

WIP: OCPBUGS-78044: Improve WatchList test robustness#2622

Open
jacobsee wants to merge 1 commit intoopenshift:masterfrom
jacobsee:watchlist-test-use-labelselector
Open

WIP: OCPBUGS-78044: Improve WatchList test robustness#2622
jacobsee wants to merge 1 commit intoopenshift:masterfrom
jacobsee:watchlist-test-use-labelselector

Conversation

@jacobsee
Copy link
Copy Markdown
Member

@jacobsee jacobsee commented Mar 9, 2026

What type of PR is this?

/kind bug
/kind flake

What this PR does / why we need it:

The WatchList test [sig-api-machinery] API Streaming (aka. WatchList) [FeatureGate:WatchList] [Beta] should be requested by metadatainformer when WatchListClient is enabled works by fetching an expected (initial) state of secrets, starting an informer, and polling until context timeout for the informer to converge to the expected state. If any secret in the namespace changes while the test is running, they never converge, and the test times out. This change limits the secrets we’re listing to just the ones relevant to the test.

NOTE: This PR is in this state for testing. Once I'm confident this is the issue, this should really be submitted upstream.

Which issue(s) this PR is related to:

Special notes for your reviewer:

Does this PR introduce a user-facing change?


Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:


Summary by CodeRabbit

  • Chores
    • Updated secret metadata filtering to apply consistent label-based selectors across all informer and list operations. This standardization ensures synchronized and accurate secret retrieval throughout the system, improving the reliability and consistency of secret management operations by aligning filtering behavior across all access points.

The WatchList test “[sig-api-machinery] API Streaming (aka. WatchList) [FeatureGate:WatchList] [Beta] should be requested by metadatainformer when WatchListClient is enabled” works by fetching an expected (initial) state of secrets, starting an informer, and polling until context timeout for the informer to converge to the expected state. If any secret in the namespace changes while the test is running, they never converge, and the test times out. This change limits the secrets we’re listing to just the ones relevant to the test.
@openshift-ci-robot openshift-ci-robot added the backports/unvalidated-commits Indicates that not all commits come to merged upstream PRs. label Mar 9, 2026
@openshift-ci openshift-ci bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/bug Categorizes issue or PR as related to a bug. labels Mar 9, 2026
@openshift-ci-robot openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Mar 9, 2026
@openshift-ci-robot
Copy link
Copy Markdown

@jacobsee: This pull request references Jira Issue OCPBUGS-78044, which is invalid:

  • expected the bug to target the "4.22.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

What type of PR is this?

/kind bug
/kind flake

What this PR does / why we need it:

The WatchList test [sig-api-machinery] API Streaming (aka. WatchList) [FeatureGate:WatchList] [Beta] should be requested by metadatainformer when WatchListClient is enabled works by fetching an expected (initial) state of secrets, starting an informer, and polling until context timeout for the informer to converge to the expected state. If any secret in the namespace changes while the test is running, they never converge, and the test times out. This change limits the secrets we’re listing to just the ones relevant to the test.

NOTE: This PR is in this state for testing. Once I'm confident this is the issue, this should really be submitted upstream.

Which issue(s) this PR is related to:

Special notes for your reviewer:

Does this PR introduce a user-facing change?


Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:


Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot added the kind/flake Categorizes issue or PR as related to a flaky test. label Mar 9, 2026
@openshift-ci-robot
Copy link
Copy Markdown

@jacobsee: the contents of this pull request could not be automatically validated.

The following commits could not be validated and must be approved by a top-level approver:

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 9, 2026

Walkthrough

This change adds a label selector filter watchlist=true to metadata informer setup and updates List calls on the metadata client for secrets to include the same label filter in an e2e test file.

Changes

Cohort / File(s) Summary
Label Selector Filtering
test/e2e/apimachinery/watchlist.go
Adds LabelSelector: "watchlist=true" filter to metadata informer tweakListOptions and updates all metadata client List calls to include the same label selector.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~7 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Test Structure And Quality ⚠️ Warning Test file lacks meaningful assertion error messages; 35 of 37 ExpectNoError calls are missing messages and no explicit setup/teardown blocks. Add contextual error messages to all assertions and implement explicit BeforeEach/AfterEach blocks for structured resource management.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title references an issue tracker code (OCPBUGS-78044) and describes the intent to improve test robustness, which aligns with the changeset that adds label selector filtering to a flake-prone WatchList test.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Stable And Deterministic Test Names ✅ Passed All Ginkgo test titles in watchlist.go follow stable and deterministic naming guidelines without dynamic values.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Tip

Try Coding Plans. Let us write the prompt for your AI agent so you can ship faster (with fewer bugs).
Share your feedback on Discord.


Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci openshift-ci bot requested review from jerpeter1 and p0lyn0mial March 10, 2026 00:00
@openshift-ci
Copy link
Copy Markdown

openshift-ci bot commented Mar 10, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: jacobsee
Once this PR has been reviewed and has the lgtm label, please assign deads2k for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot
Copy link
Copy Markdown

@jacobsee: This pull request references Jira Issue OCPBUGS-78044, which is invalid:

  • expected the bug to target the "4.22.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

What type of PR is this?

/kind bug
/kind flake

What this PR does / why we need it:

The WatchList test [sig-api-machinery] API Streaming (aka. WatchList) [FeatureGate:WatchList] [Beta] should be requested by metadatainformer when WatchListClient is enabled works by fetching an expected (initial) state of secrets, starting an informer, and polling until context timeout for the informer to converge to the expected state. If any secret in the namespace changes while the test is running, they never converge, and the test times out. This change limits the secrets we’re listing to just the ones relevant to the test.

NOTE: This PR is in this state for testing. Once I'm confident this is the issue, this should really be submitted upstream.

Which issue(s) this PR is related to:

Special notes for your reviewer:

Does this PR introduce a user-facing change?


Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:


Summary by CodeRabbit

  • Chores
  • Updated secret metadata filtering to apply consistent label-based selectors across all informer and list operations. This standardization ensures synchronized and accurate secret retrieval throughout the system, improving the reliability and consistency of secret management operations by aligning filtering behavior across all access points.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
test/e2e/apimachinery/watchlist.go (1)

106-108: Extract the watchlist selector into a shared constant.

"watchlist=true" is now duplicated across the informer, the direct List calls, and the request assertion helper. Hoisting the label key/value/selector into one place will make this test less brittle if the filter ever changes.

♻️ Suggested cleanup
+const (
+	watchlistLabelKey      = "watchlist"
+	watchlistLabelValue    = "true"
+	watchlistLabelSelector = watchlistLabelKey + "=" + watchlistLabelValue
+)
+
 ...
 			func(options *metav1.ListOptions) {
-				options.LabelSelector = "watchlist=true"
+				options.LabelSelector = watchlistLabelSelector
 			},
 ...
-		expectedSecrets, err := metadataClient.Resource(v1.SchemeGroupVersion.WithResource("secrets")).Namespace(f.Namespace.Name).List(ctx, metav1.ListOptions{LabelSelector: "watchlist=true"})
+		expectedSecrets, err := metadataClient.Resource(v1.SchemeGroupVersion.WithResource("secrets")).Namespace(f.Namespace.Name).List(ctx, metav1.ListOptions{LabelSelector: watchlistLabelSelector})
 ...
-		expectedSecrets, err = metadataClient.Resource(v1.SchemeGroupVersion.WithResource("secrets")).Namespace(f.Namespace.Name).List(ctx, metav1.ListOptions{LabelSelector: "watchlist=true"})
+		expectedSecrets, err = metadataClient.Resource(v1.SchemeGroupVersion.WithResource("secrets")).Namespace(f.Namespace.Name).List(ctx, metav1.ListOptions{LabelSelector: watchlistLabelSelector})
 ...
-			Labels: map[string]string{"watchlist": "true"},
+			Labels: map[string]string{watchlistLabelKey: watchlistLabelValue},
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/e2e/apimachinery/watchlist.go` around lines 106 - 108, Create a shared
constant for the watchlist label selector (e.g., WatchlistLabelSelector) and
replace all hard-coded occurrences of "watchlist=true" with that constant:
update the metav1.ListOptions lambda where options.LabelSelector is set, the
direct List(...) calls, and the request assertion helper that validates the
selector; keep the constant close to the test package (same file or a shared
test helper) and reference it by name wherever LabelSelector is compared or
assigned so the selector value is maintained in one place.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@test/e2e/apimachinery/watchlist.go`:
- Around line 106-108: Create a shared constant for the watchlist label selector
(e.g., WatchlistLabelSelector) and replace all hard-coded occurrences of
"watchlist=true" with that constant: update the metav1.ListOptions lambda where
options.LabelSelector is set, the direct List(...) calls, and the request
assertion helper that validates the selector; keep the constant close to the
test package (same file or a shared test helper) and reference it by name
wherever LabelSelector is compared or assigned so the selector value is
maintained in one place.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 6764aef9-d11e-405c-ab5f-a0f2ba8f7b9c

📥 Commits

Reviewing files that changed from the base of the PR and between ac14da2 and 9f30989.

📒 Files selected for processing (1)
  • test/e2e/apimachinery/watchlist.go

@jacobsee
Copy link
Copy Markdown
Member Author

/retest

@jacobsee
Copy link
Copy Markdown
Member Author

/payload-aggregate periodic-ci-openshift-release-main-ci-4.22-upgrade-from-stable-4.21-e2e-gcp-ovn-rt-upgrade

@openshift-ci
Copy link
Copy Markdown

openshift-ci bot commented Mar 10, 2026

@jacobsee: it appears that you have attempted to use some version of the payload command, but your comment was incorrectly formatted and cannot be acted upon. See the docs for usage info.

@jacobsee
Copy link
Copy Markdown
Member Author

/payload-aggregate periodic-ci-openshift-release-main-ci-4.22-upgrade-from-stable-4.21-e2e-gcp-ovn-rt-upgrade 10

@openshift-ci
Copy link
Copy Markdown

openshift-ci bot commented Mar 10, 2026

@jacobsee: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-main-ci-4.22-upgrade-from-stable-4.21-e2e-gcp-ovn-rt-upgrade

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/cfef87d0-1ca1-11f1-91fe-856567d27159-0

@jacobsee
Copy link
Copy Markdown
Member Author

/payload-aggregate periodic-ci-openshift-release-main-ci-4.22-upgrade-from-stable-4.21-e2e-gcp-ovn-rt-upgrade 10

@openshift-ci
Copy link
Copy Markdown

openshift-ci bot commented Mar 16, 2026

@jacobsee: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-main-ci-4.22-upgrade-from-stable-4.21-e2e-gcp-ovn-rt-upgrade

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/74497360-215a-11f1-9801-a8403ecd0929-0

@sjenning
Copy link
Copy Markdown

/test e2e-aws-ovn-hypershift

@openshift-ci
Copy link
Copy Markdown

openshift-ci bot commented Mar 25, 2026

@jacobsee: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-ovn-runc 9f30989 link false /test e2e-aws-ovn-runc
ci/prow/e2e-aws-ovn-hypershift 9f30989 link true /test e2e-aws-ovn-hypershift

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backports/unvalidated-commits Indicates that not all commits come to merged upstream PRs. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. kind/bug Categorizes issue or PR as related to a bug. kind/flake Categorizes issue or PR as related to a flaky test.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants