Skip to content

github actions: compare test results only against merged PRs#920

Open
bmastbergen wants to merge 1 commit intomainfrom
{bmastbergen}_main
Open

github actions: compare test results only against merged PRs#920
bmastbergen wants to merge 1 commit intomainfrom
{bmastbergen}_main

Conversation

@bmastbergen
Copy link
Collaborator

@bmastbergen bmastbergen commented Feb 25, 2026

KERNEL-636

Previously the workflow compared test results against any successful run targeting the same base branch. This could include experimental branches that were never merged or PRs that were later rejected.

Now the comparison logic checks if each candidate run's PR was actually merged before using it as a baseline. This ensures comparisons are made against stable, merged code that made it into the base branch.

The workflow still falls back gracefully if no merged PRs are found (comparison is skipped but PR creation proceeds).

Here is a PR generated with these changes:
#919

Copilot AI review requested due to automatic review settings February 25, 2026 15:28
@bmastbergen bmastbergen requested a review from a team February 25, 2026 15:29
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the kernel multiarch GitHub Actions workflow so kselftest comparisons use baselines only from workflow runs associated with PRs that were actually merged into the target base branch, avoiding comparisons against abandoned/experimental branches.

Changes:

  • Adjust baseline selection to require a merged PR before downloading prior kselftest artifacts.
  • Update workflow messaging/warnings to reflect “merged PR baseline” behavior.
  • Export the baseline PR number (BASELINE_PR) when a baseline is found.
Comments suppressed due to low confidence (1)

.github/workflows/kernel-build-and-test-multiarch.yml:394

  • This loop can make up to 50 gh pr list calls (plus artifact API calls) before finding a baseline, which may exceed the 3-minute timeout and/or hit API rate limits. Consider fetching run details once (include event/pullRequests in gh run list --json ..., or query the run via gh api only for candidate branches) and only calling the artifacts endpoint after confirming a specific PR is merged.
        # Get last 50 successful workflow runs (cast a wider net to find PRs targeting this base)
        # We need to check each run to see if it targets the same base branch AND was merged
        SUCCESSFUL_RUNS=$(gh run list \
          --workflow kernel-build-and-test-multiarch.yml \
          --status success \
          --limit 50 \
          --json databaseId,headBranch,createdAt)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 420 to 426
# Check if the PR from this branch was merged
PR_INFO=$(gh pr list --head "$HEAD_BRANCH" --state merged --json number,mergedAt --jq '.[0]' 2>/dev/null || echo "")

if [ -n "$PR_INFO" ] && [ "$PR_INFO" != "null" ]; then
PR_NUMBER=$(echo "$PR_INFO" | jq -r '.number')
MERGED_AT=$(echo "$PR_INFO" | jq -r '.mergedAt')
echo "Found merged PR #$PR_NUMBER from branch $HEAD_BRANCH (merged: $MERGED_AT, targets: $BASE_BRANCH)"
Copy link

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The merged-check is based only on --head "$HEAD_BRANCH", which can misclassify runs as “merged PR” even when the workflow run was triggered by a later push to the same branch (after the PR merged) or by a different PR that reused the branch name. This can pick a baseline that includes commits that never landed in the base branch. Prefer deriving the PR number from the workflow run itself (e.g., via the run payload pull_requests[0].number / event == pull_request) and then verify that specific PR is merged before using its artifacts.

Copilot uses AI. Check for mistakes.
roxanan1996
roxanan1996 previously approved these changes Feb 25, 2026
Previously the workflow compared test results against any successful run
targeting the same base branch. This could include experimental branches
that were never merged or PRs that were later rejected.

Now the comparison logic checks if each candidate run's PR was actually
merged before using it as a baseline. This ensures comparisons are made
against stable, merged code that made it into the base branch.

The workflow still falls back gracefully if no merged PRs are found
(comparison is skipped but PR creation proceeds).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

4 participants