Skip to content

[Feature] Introduce CI integration tests with provider health gates #47

@mbe24

Description

@mbe24

Problem

External integration tests currently mix very stable fixtures and volatile dependencies in one lane. This creates noisy failures from missing/expired credentials and endpoint drift, while also making it harder to tell infrastructure issues from adapter regressions.

Design

Introduce two external-dependency CI tiers plus workflow-level provider health gates:

  1. Tier A (blocking): stable external fixtures.
  2. Tier B (advisory): token and instance dependent providers.
  3. Per-provider smoke checks (env + auth/reachability) run before full provider test slices.

Initial mapping:

  1. Tier A: GitHub, GitLab.
  2. Tier B: Jira, Bitbucket Cloud.

Scope

  • Add dedicated external integration workflow with per-provider jobs.
  • Add workflow-level health/auth gates before full provider test slices.
  • Keep Tier A blocking and Tier B advisory.
  • Execute provider-specific integration slices by name prefix filters.
  • Document fixture and credential requirements for CI and local runs.

Boundary

  • No adapter behavior changes in this issue.
  • No migration of integration tests into multiple files in this phase.
  • No immediate promotion of Jira/Bitbucket to blocking.

Acceptance Criteria

  • Tier A provider jobs block PRs and fail clearly on broken fixtures/auth.
  • Tier B provider jobs run as advisory and do not block merges.
  • Each provider job reports whether failure was due to missing env, invalid token, endpoint health, or test assertions.
  • External workflow can be run manually and on schedule with predictable behavior.
  • Core blocking CI jobs remain unchanged and stable.

Context

The project now has broader provider coverage and more external integration tests. A tiered strategy improves reliability and triage quality while preserving meaningful blocking checks for stable providers.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions