Skip to content

fix(ci): correct needs: parallel: matrix: syntax for downstream job dependencies#182

Merged
wdconinc merged 2 commits intomasterfrom
copilot/fix-gitlab-ci-needs-syntax
Mar 8, 2026
Merged

fix(ci): correct needs: parallel: matrix: syntax for downstream job dependencies#182
wdconinc merged 2 commits intomasterfrom
copilot/fix-gitlab-ci-needs-syntax

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 5, 2026

GitLab CI needs: referencing a specific parallel matrix job instance requires scalar values — using YAML list notation causes GitLab to fail resolving the dependency, so downstream jobs run immediately without waiting for their required container images.

Changes

  • 6 jobs in .gitlab-ci.yml had ENV and BUILD_TYPE changed from list to scalar syntax in their needs: parallel: matrix: blocks:
    • user_spack_environment (needs eic_ci, default)
    • .cuda (needs eic_dev_cuda, default)
    • eic_xl:singularity:default / eic_xl:singularity:nightly
    • .benchmarks:default / .benchmarks:nightly

Before:

needs:
  - job: eic
    parallel:
      matrix:
        - BUILD_IMAGE: eic_
          ENV:
          - ci
          BUILD_TYPE:
          - default

After:

needs:
  - job: eic
    parallel:
      matrix:
        - BUILD_IMAGE: eic_
          ENV: ci
          BUILD_TYPE: default
          BUILDER_IMAGE: debian_stable_base
          RUNTIME_IMAGE: debian_stable_base
          PLATFORM: linux/amd64
Original prompt

Problem

The .gitlab-ci.yml file has incorrect syntax in the needs: sections for several jobs that depend on specific parallel matrix job instances. The current syntax uses list notation for matrix variables, which doesn't properly specify a single job dependency:

Current (incorrect):

needs:
  - job: eic
    parallel:
      matrix:
        - BUILD_IMAGE: eic_
          ENV:
          - ci
          BUILD_TYPE:
          - default

This causes jobs to execute without waiting for their required container images to be built, leading to failures.

Expected (correct):

needs:
  - job: eic
    parallel:
      matrix:
        - BUILD_IMAGE: eic_
          ENV: ci
          BUILD_TYPE: default
          BUILDER_IMAGE: debian_stable_base
          RUNTIME_IMAGE: debian_stable_base
          PLATFORM: linux/amd64

Jobs Affected

The following jobs in .gitlab-ci.yml need to be fixed:

  1. user_spack_environment - needs eic_ci (BUILD_TYPE: default, ENV: ci)
  2. .cuda - needs eic_dev_cuda (BUILD_TYPE: default, ENV: cuda)
  3. eic_xl:singularity:default - needs eic_xl (BUILD_TYPE: default, ENV: xl)
  4. eic_xl:singularity:nightly - needs eic_xl (BUILD_TYPE: nightly, ENV: xl)
  5. .benchmarks:default - needs eic_ci (BUILD_TYPE: default, ENV: ci)
  6. .benchmarks:nightly - needs eic_ci (BUILD_TYPE: nightly, ENV: ci)

Solution

For each affected job, change the needs: parallel: matrix: section from list syntax to single value syntax, and include all matrix variables to uniquely identify the specific parallel job instance to depend on.

Files to Modify

  • .gitlab-ci.yml

References

This pull request was created from Copilot chat.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

@wdconinc
Copy link
Copy Markdown
Contributor

wdconinc commented Mar 5, 2026

This PR intends to solve incorrect (early) job instantiations in e.g. https://eicweb.phy.anl.gov/containers/eic_container/-/jobs/7378982.

Co-authored-by: wdconinc <4656391+wdconinc@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix syntax in needs sections of .gitlab-ci.yml fix(ci): correct needs: parallel: matrix: syntax for downstream job dependencies Mar 5, 2026
@wdconinc wdconinc marked this pull request as ready for review March 5, 2026 13:34
Copilot AI review requested due to automatic review settings March 5, 2026 13:34
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes GitLab CI needs: dependencies on specific parallel:matrix job instances so downstream jobs correctly wait for the exact container image build they require (instead of running immediately due to unresolved dependencies).

Changes:

  • Updates 6 needs: parallel: matrix: blocks in .gitlab-ci.yml to use scalar values for ENV and BUILD_TYPE (e.g., ENV: ci), matching GitLab’s expected syntax for targeting a single matrix-expanded job.
  • Keeps the full matrix key set (e.g., BUILDER_IMAGE, RUNTIME_IMAGE, PLATFORM) to uniquely identify the intended upstream build instance.

@wdconinc wdconinc requested a review from veprbl March 5, 2026 23:55
@wdconinc wdconinc enabled auto-merge (squash) March 7, 2026 00:07
Copy link
Copy Markdown
Member

@veprbl veprbl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good, of course.

@wdconinc wdconinc merged commit e536425 into master Mar 8, 2026
33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants