github actions: compare test results only against merged PRs#920
github actions: compare test results only against merged PRs#920bmastbergen wants to merge 1 commit intomainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
Updates the kernel multiarch GitHub Actions workflow so kselftest comparisons use baselines only from workflow runs associated with PRs that were actually merged into the target base branch, avoiding comparisons against abandoned/experimental branches.
Changes:
- Adjust baseline selection to require a merged PR before downloading prior kselftest artifacts.
- Update workflow messaging/warnings to reflect “merged PR baseline” behavior.
- Export the baseline PR number (
BASELINE_PR) when a baseline is found.
Comments suppressed due to low confidence (1)
.github/workflows/kernel-build-and-test-multiarch.yml:394
- This loop can make up to 50
gh pr listcalls (plus artifact API calls) before finding a baseline, which may exceed the 3-minute timeout and/or hit API rate limits. Consider fetching run details once (includeevent/pullRequestsingh run list --json ..., or query the run viagh apionly for candidate branches) and only calling the artifacts endpoint after confirming a specific PR is merged.
# Get last 50 successful workflow runs (cast a wider net to find PRs targeting this base)
# We need to check each run to see if it targets the same base branch AND was merged
SUCCESSFUL_RUNS=$(gh run list \
--workflow kernel-build-and-test-multiarch.yml \
--status success \
--limit 50 \
--json databaseId,headBranch,createdAt)
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # Check if the PR from this branch was merged | ||
| PR_INFO=$(gh pr list --head "$HEAD_BRANCH" --state merged --json number,mergedAt --jq '.[0]' 2>/dev/null || echo "") | ||
|
|
||
| if [ -n "$PR_INFO" ] && [ "$PR_INFO" != "null" ]; then | ||
| PR_NUMBER=$(echo "$PR_INFO" | jq -r '.number') | ||
| MERGED_AT=$(echo "$PR_INFO" | jq -r '.mergedAt') | ||
| echo "Found merged PR #$PR_NUMBER from branch $HEAD_BRANCH (merged: $MERGED_AT, targets: $BASE_BRANCH)" |
There was a problem hiding this comment.
The merged-check is based only on --head "$HEAD_BRANCH", which can misclassify runs as “merged PR” even when the workflow run was triggered by a later push to the same branch (after the PR merged) or by a different PR that reused the branch name. This can pick a baseline that includes commits that never landed in the base branch. Prefer deriving the PR number from the workflow run itself (e.g., via the run payload pull_requests[0].number / event == pull_request) and then verify that specific PR is merged before using its artifacts.
Previously the workflow compared test results against any successful run targeting the same base branch. This could include experimental branches that were never merged or PRs that were later rejected. Now the comparison logic checks if each candidate run's PR was actually merged before using it as a baseline. This ensures comparisons are made against stable, merged code that made it into the base branch. The workflow still falls back gracefully if no merged PRs are found (comparison is skipped but PR creation proceeds).
3430536 to
c686dce
Compare
KERNEL-636
Previously the workflow compared test results against any successful run targeting the same base branch. This could include experimental branches that were never merged or PRs that were later rejected.
Now the comparison logic checks if each candidate run's PR was actually merged before using it as a baseline. This ensures comparisons are made against stable, merged code that made it into the base branch.
The workflow still falls back gracefully if no merged PRs are found (comparison is skipped but PR creation proceeds).
Here is a PR generated with these changes:
#919