Skip to content

Add regression test missing CI check #45

@Svtter

Description

@Svtter

Background

Currently we have two AI-powered PR checks in CI:

  • review: Code quality review (checks for bugs, style, security, performance)
  • feature-missing: Compares PR implementation against linked issue specs to find missing features

We need a third check: regression-test-missing. When a PR fixes a bug or modifies existing behavior, it should include regression tests to prevent the same issue from recurring.

Existing Pattern

Both current checks follow the same structure:

  1. A composite action directory (review/, feature-missing/) with action.yml
  2. A workflow file (.github/workflows/review.yml, .github/workflows/feature-missing.yml) triggered on PR events
  3. The action wraps github-run-opencode/run-github-opencode.py with a specialized prompt

Deliverables

1. regression-test-missing/ composite action

An independent composite action (same level as review/ and feature-missing/) that:

  • Detects bugfix/refactor PRs: Identifies when a PR is fixing a bug, addressing a crash, or modifying existing behavior (not just adding new features)
  • Checks for regression tests: Verifies if the PR includes tests that would catch the same bug/issue if it were reintroduced
  • Distinguishes scenarios:
    • Bugfix without regression test → flag as missing
    • Pure feature addition (no existing behavior change) → skip or pass
    • Bugfix with adequate test coverage → pass
  • Output format (consistent with other checks, in Chinese):
    • First line: "无需回归测试" or "缺少回归测试"
    • Summary of analysis
    • If missing: specific suggestions on what regression tests should be added

2. Integrated workflow action (pr-checks/ or similar)

A new composite action that combines all three checks in one workflow:

  • 3 parallel jobs: review, feature-missing, regression-test-missing
  • Each job calls the corresponding independent composite action
  • Shares opencode install cache across jobs (方案 C)
  • Users can adopt this single action instead of configuring 3 separate workflows

Implementation Checklist

  • Create regression-test-missing/action.yml composite action (modeled after feature-missing/action.yml)
  • Create integrated composite action directory with action.yml containing 3 parallel jobs
  • Craft prompt for regression-test-missing that instructs the AI to:
    • Look for bugfixes, crash fixes, behavior changes in the PR diff
    • Check if corresponding regression/reproducer tests exist
    • Suggest specific test cases if missing
  • Update .github/workflows/ci.yml smoke test to cover new action
  • Update README documentation if actions are documented there

Acceptance Criteria

  • regression-test-missing composite action works standalone
  • Integrated workflow action runs all 3 checks in parallel
  • Check correctly identifies bugfix PRs that lack regression tests
  • Check passes for PRs that already include adequate regression tests
  • Check passes (or is neutral) for pure feature-addition PRs with no behavior changes
  • Follows the same architectural pattern as review and feature-missing

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions