Add subdir-aware inputs to standard reusables (v2.2.0)#107
Merged
Conversation
mtfishman
changed the title
mtfishman
changed the title
Three small additive inputs that let consumers with in-tree subpackages (notably ITensors.jl + NDTensors) drop their custom workflow files in favor of the standard reusables: - CompatHelper.yml `subdirs`: newline-separated list of subdirectories whose Project.toml should be checked for compat bumps. Default preserves the historical hardcoded list (root + docs + examples + test). - Documentation.yml `extra-dev-paths`: newline-separated list of additional local package paths to Pkg.develop() before the docs build. Mirrors the existing input on IntegrationTest.yml. - Tests.yml `test-args`: surfaces julia-actions/julia-runtest's `test_args` input for callers whose runtests.jl reads ARGS instead of the GROUP env-var. All three have defaults that preserve current behavior; existing consumers see no change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors the extra-dev-paths inputs on Documentation.yml and IntegrationTest.yml. Needed for repos where the package's in-tree subpackage must shadow any registered version during testing (notably ITensors.jl + ./NDTensors). A new pre-buildpkg step runs `Pkg.develop` on each path into the package's active project, so subsequent buildpkg + julia-runtest see the in-tree subpackage instead of resolving the registered one. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lets multi-package repos (notably ITensors.jl + NDTensors) auto-register each in-tree package by invoking the Registrator reusable once per subdir, typically via a matrix. The subdir is prepended to all package-relative paths and surfaces in the JuliaRegistrator commit comment as `@JuliaRegistrator register subdir=<subdir>` so the registrar picks up the right Project.toml. Default empty preserves the existing single-package root behavior; no behavior change for current consumers. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…port
- Tests.yml: the Pkg.develop pre-step previously hardcoded
Pkg.activate(".") which would shadow any caller-supplied project
input. Now reads inputs.project so the dev paths are added to the
active project being tested (matters when project=NDTensors).
- VersionCheck.yml: add `subdirs` input. With it set, the workflow
classifies changed files into per-project scopes (root + each
subdir) and validates that each substantively-changed project's
Project.toml was bumped. Backward-compatible: empty subdirs reduces
to the existing single-project root check, and the metadata-only
PR case still passes without bump (root scope is empty).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous version did `all_pass = true` then `all_pass &= ...` inside a `for` loop. Julia's hard-scope rule for top-level `for` loops in a script context shadows outer variables on assignment, so the `&=` tried to write to a fresh local `all_pass` without first reading it, producing `UndefVarError: \`all_pass\` not defined in local scope`. Replace with a comprehension that evaluates every project (so all errors surface before failing) plus an `all(results)` check. Caught by the new VersionCheck running on ITensors.jl#1743. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
mtfishman
force-pushed
the
mf/subdir-inputs
branch
from
baf7b23 to
f5f5b3d
Compare
Dropped from #107's scope after redesign. Multi-package consumers (ITensors.jl + NDTensors) now use one workflow file per package rather than a single Tests.yml that cross-`Pkg.develop`s a sibling subpackage. With the per-package shape, no Tests.yml caller in the ecosystem needs `extra-dev-paths` — each package runs its own tests against registered deps for the others. `Tests.test-args` is kept (NDTensors's runtests reads ARGS for group selection). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Dropped from #107's scope after redesign. Under the per-package workflow shape (Model 2), each package's docs build runs against its own deps; no caller in the ecosystem needs to develop a sibling subpackage before the docs build. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Dropped from #107's scope after redesign. Under the per-package workflow shape (Model 2), each package gets its own CompatHelper.yml caller and runs the standard single-subpackage default. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Dropped per-project subdir-aware classification from #107's scope after redesign. Under the per-package workflow shape (Model 2), each package gets its own VersionCheck.yml caller and runs the standard single-project check (using the existing `classify-pr` composite action to skip non-substantive PRs). Also reverts the Julia hard-scope bug fix from f5f5b3d, which existed only in the per-project results loop being removed here. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lets multi-package repos run an IntegrationTest matrix with an in-tree subpackage as the package-under-test. Empty default preserves the existing root-of-repo behavior; setting `subdir: "NDTensors"` for example makes the workflow `Pkg.develop` from `./NDTensors` instead of `.`, so the matrix tests downstream consumers against the in-tree subpackage. Used in ITensors.jl's NDTensors-IntegrationTest caller (#1743) so NDTensors changes get tested against ITensors and its downstream consumers. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a `classify` job that runs the existing classify-pr composite action up front. The matrix `integration-test` job now needs `classify` and runs only when `substantive == 'true'` (or the new `run-on-nonsubstantive` opt-out is set). The `gate` job short-circuits to success when the PR is non-substantive, so the required `IntegrationTest` check still reports green. Default-on saves CI compute on metadata-only PRs (`.github/**`, `.pre-commit-config.yaml`, `.gitignore`, `LICENSE`) where downstream tests can't be affected. Callers wanting the previous always-run behavior set `run-on-nonsubstantive: true`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drop the multi-sentence rationale paragraphs from `subdir` and `run-on-nonsubstantive` — match the terse style of the other inputs in this file (e.g. `run-on-draft`, `extra-dev-paths`). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
mtfishman
changed the title
mtfishman
changed the title
Adds two mutually-exclusive optional inputs to scope the substantive-detection per package in multi-package repos: - `subdir` — positive scope. A file is substantive iff it lies under `<subdir>/`. Used by callers that target a single in-tree subpackage (e.g. NDTensors-IntegrationTest with `subdir: NDTensors`). - `exclude-subdirs` — negative scope at root. Newline-separated list of sibling subdirectories to exclude. A file is substantive iff it lies outside the standard root-metadata set (`.github/**`, `.pre-commit-config.yaml`, `.gitignore`, `LICENSE`) AND outside any listed sibling. Used by callers that target the root package and need to ignore changes confined to a sibling subpackage (e.g. ITensors-IntegrationTest with `exclude-subdirs: NDTensors`). Both empty preserves the current global behavior — single-package consumers see no change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Renames the activation knob from `project` to `subdir` to match the
naming used by other reusables (Registrator, IntegrationTest,
forthcoming VersionCheck). When `subdir` is set:
- Julia activates `<subdir>` for both buildpkg and runtest.
- classify-pr scopes the substantive-PR check to files under
`<subdir>/`.
Adds `exclude-subdirs` for the root-scope case (newline-separated
sibling subpackages to ignore in the substantive-PR check).
Both inputs default empty. When both are empty, classify-pr still
runs (in its global mode) but Tests ignores the result and runs on
every PR — same behavior single-package consumers see today.
Drops the legacy `project` input. ITensors.jl is the only known
ecosystem consumer setting `project` to a non-default value, and
that caller is being updated as part of this work.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Keeps `project` available so existing single-package consumers pinned to `@v2` see no behavior change after v2.2.0 ships. New multi-package shape uses `subdir`; `subdir` takes precedence when non-empty. Plan to remove `project` in a future major release. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds `exclude-subdirs` input and wires both `subdir` and `exclude-subdirs` through to the `classify-pr` step. With this, NDTensors-IntegrationTest (`subdir: NDTensors`) classifies ITensors-only PRs as non-substantive within scope and skips, and ITensors-IntegrationTest (`exclude-subdirs: NDTensors`) classifies NDTensors-only PRs the same way. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`subdir` controls which Project.toml the version-bump check reads (empty = root, "NDTensors" = NDTensors/Project.toml, etc.). Both `subdir` and `exclude-subdirs` forward to classify-pr so the substantive-PR check is per-package-aware. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`subdir` becomes the canonical multi-package input. When set, it overrides both `project` (buildpkg --project) and `workspace-root` (check-compat-bounds workspace root) to point at the subpackage. `project` is marked deprecated; both remain functional for backward compat with single-package consumers. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When set, CompatHelper bumps the subpackage's Project.toml plus its docs/examples/test variants if present (e.g. with `subdir: NDTensors`, the subdirs list becomes ["NDTensors", "NDTensors/docs", "NDTensors/examples", "NDTensors/test"]). Empty default preserves the existing single-package behavior. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ite descriptions In subdir-positive scope, files under `<subdir>/` matching the same metadata-only patterns as the repo root (`.github/`, `.gitignore`, `LICENSE`, `.pre-commit-config.yaml`) now correctly count as non-substantive. Previously any change inside the subdirectory was treated as substantive — including hypothetical `<subdir>/.github/foo.yml`-style metadata changes that shouldn't trigger downstream tests. Rewrites both input descriptions to be self-contained and avoid the misleading "iff it lies under `<subdir>/`" framing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reusables that declare a job-level `permissions:` block can only narrow the consumer's workflow-level ceiling, never raise it. Single-package consumers (e.g. SparseArraysBase) grant only `contents: read` for these workflows, so the classify job's inner request for `pull-requests: read` exceeded the ceiling and GitHub rejected the workflow at startup. Dropping the inner block lets the classify job inherit the caller's permissions. The `gh api pulls/N/files` call inside classify-pr works on public repos with just `contents: read`. For private repos, callers can still grant `pull-requests: read` themselves. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…f/subdir-inputs branch The classify-pr action's subdir/exclude-subdirs inputs only exist on this branch. References pointing to the `main` branch silently ignored those inputs (returning global classification), causing per-package scoping to fail. Pinning to the `mf/subdir-inputs` branch makes the new inputs take effect. After v2.2.0 ships and the `main` branch has the updated classify-pr, a follow-up will flip these references back. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The classify-pr scoping doesn't belong inside the Tests reusable. The reusable's matrix is provided by the consumer (each leg expands to one reusable invocation), so an inner classify job runs once per matrix leg — wasteful and visually noisy in CI. Move classify-pr to the consumer side: each consumer that wants per-package skip-on-non-substantive adds its own top-level classify job and gates its matrix on the result. Single-package consumers don't need any of this. `subdir` input stays for activation. `exclude-subdirs` had no purpose without the inner classify and was dropped in the previous commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… vs. version-check
`classify-pr` action becomes a generic scope check:
- `paths` (positive scope): file triggers iff it matches one of
these paths (equals or starts-with).
- `exclude-paths` (negative scope): file does NOT trigger if it
matches one of these. Default ignores `.gitignore` and
`.pre-commit-config.yaml` (overridable).
- Mutually exclusive; positive wins with a warning if both set.
- Output: `triggers` (bool).
Reusables:
- `Tests.yml`: classify-pr step inside the matrix-leg job; all
test work gated on `steps.classify.outputs.triggers`. Per-leg
classify duplication is intentional — single-package consumers'
matrix is at the caller side, so classify must live inside each
leg to avoid a separate consumer-level job.
- `IntegrationTest.yml`: same shape — classify-pr step inside the
matrix leg. Drops the previous separate `classify` job, the
`subdir`/`exclude-subdirs` inputs (replaced by `paths`/
`exclude-paths`), the `run-on-nonsubstantive` opt-out, and the
substantive-only-by-default behavior. `subdir` is kept for its
"develop the package in this subdirectory" semantic.
- `VersionCheck.yml`: same shape; default `exclude-paths` adds
`.github` so workflow-only PRs don't require a version bump.
Net behavior:
- Single-package consumers: defaults give "any change except
pure metadata triggers tests" and "any change except metadata +
`.github` triggers version check" — matches pre-#107 behavior.
- Multi-package consumers (e.g. ITensors.jl): pass `paths` or
`exclude-paths` per package scope. Workflow-as-part-of-package
is expressed by listing `.github` in the per-package paths
(e.g. `paths: ["NDTensors", ".github"]`).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… paths When a reusable has a `subdir` input set, the package is by definition in scope for that workflow's classify-pr check (you're testing / version-checking that subdir). Auto-appending `subdir` to `paths` saves callers from repeating it. Concretely on TestsNDTensors / IntegrationTestNDTensors callers, the caller now passes `subdir: "NDTensors"` + `paths: ".github"` instead of having to spell out `paths: "NDTensors\n.github"`. On VersionCheckNDTensors the caller can drop `paths` entirely — `subdir` alone gives the right scope. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the ternary listing two parallel arrays with a single base list and a broadcasted joinpath when subdir is non-empty. Note the empty-string entry becomes `<subdir>/` (with trailing slash) under broadcasting; CompatHelper treats that the same as `<subdir>` when reading `<subdir>/Project.toml`, so no functional change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reusables that auto-append `subdir` into `paths` (Tests.yml, IntegrationTest.yml, VersionCheck.yml) produce a list of all-empty lines when both `subdir` and the caller's `paths` are empty. Bash's [ -n ] would otherwise read that whitespace-only string as "positive scope set", flipping the action into positive-scope mode with an empty list — so no files match, triggers=false, and the caller's exclude-paths default is silently bypassed. Normalize at the top of the script: any input that contains only empty lines is treated as empty. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- README: add subdir / paths / exclude-paths rows to the input tables for Tests, IntegrationTest, VersionCheck, CheckCompatBounds, CompatHelper, and Registrator. Mark `project` as a deprecated alias for `subdir` on Tests and CheckCompatBounds. Rewrite the VersionCheck "non-substantive PRs" prose to describe the new paths / exclude-paths model. - Tests / IntegrationTest / VersionCheck reusables: flip the inner `classify-pr` reference from the development branch self-reference back to the main branch for the post-merge state. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds subdir-aware inputs to the standard ITensorActions reusables so
multi-package repositories (e.g. ITensors.jl with its in-tree NDTensors
subpackage) can use them directly instead of maintaining custom workflow
YAML. Also makes IntegrationTest skip metadata-only PRs by default,
saving CI compute ecosystem-wide.
Backward-compatible — single-package consumers pinned to
v2see nobehavior change, smoke-tested via SparseArraysBase.jl#179.
Shipping as
v2.2.0.Test plan
v2.2.0at merge commit, force-updatev2.