[BUG] Fortran source files don't propagate through `library-manifest` dependency resolution outside `task: 'build'`

[batpigandme]

Description

Error is encountered when a Fortran benchmark for a Level-2+ BLAS routine whose Fortran source declares an EXTERNAL Fortran dependency on another stdlib package (e.g., dgemm calling xerbla, dger calling xerbla). The benchmark fails to link with a missing-symbol error for the dependency.

The underlying defect is in @stdlib/utils/library-manifest. When the resolver is queried with any task other than build (notably task: 'benchmark'), Fortran source files belonging to dependencies are not surfaced.

Fortran sources are currently declared only inside the build task entry of each Fortran-using package's manifest — so when library-manifest walks the dependency tree under task: 'benchmark', it merges each dependency's benchmark entry, which contains only C sources, and the dependency's .f files never appear in the merged result.

For example, in blas/base/xerbla/manifest.json:

• task: 'build', os: 'linux', blas: '', wasm: false → src: ["./src/xerbla.f", "./src/xerbla.c"]
• task: 'benchmark', os: 'linux', blas: '', wasm: false → src: ["./src/xerbla.c"] (no .f)

The native-addon build path works bc binding.gyp/include.gypi calls library-manifest with no explicit task, defaulting to task: 'build', where the .f declarations exist.

The same condition silently affects every merged Fortran benchmark for a Level-2+ BLAS routine that exercises a cross-package Fortran symbol. Level-1 BLAS Fortran benchmarks (e.g., daxpy, dscal, dasum) have no cross-package Fortran dependencies and link cleanly, which is why the gap has not surfaced more broadly.

The recurring local fix is a Makefile patch in the consuming package that hardcodes a relative path to a sibling package's .f file (e.g., ../../../xerbla/src/xerbla.f inside dger's benchmark Makefile). This couples one package to another package's file-tree layout. If xerbla is ever moved within the package tree (e.g., to @stdlib/blas/base/utils/xerbla), every consumer's hardcoded path silently breaks, and a contributor performing the migration has no way to discover the breakage short of running every affected benchmark. The manifest-based @stdlib/... reference model exists precisely so packages don't reach into each other's file trees this way. See @kgryte's comment on PR #11332.

A second, downstream contributor to the symptom is that tools/scripts/compile_fortran_benchmark only resolves include directories from the manifest, while its C counterpart tools/scripts/compile_c_benchmark resolves include, src, libraries, and libpath. Patching only this wrapper might be too narrow, since it wouldn't address future Fortran-linking contexts that go through library-manifest.

Per office hours discussion, the fix should live in @stdlib/utils/library-manifest so that Fortran source files are resolved uniformly when resolving a package's dependency tree, regardless of the task the caller queried.

Once the resolver is correct, downstream cleanup for the Fortran benchmark wrapper in tools/scripts/ should consume the resolved source list (mirroring its C counterpart), and the merged Fortran benchmark Makefiles' hardcoded SOURCE_FILES defaults should match the C-benchmark Makefile convention.

Related Issues

Related PRs #11332 and #11333.

Questions

No.

Demo

No response

Reproduction

# In a checkout of stdlib develop:
cd lib/node_modules/@stdlib/blas/base/dger/benchmark/fortran
make

Expected Results

Benchmark links successfully; xerbla resolves via the manifest dependency graph (dger's manifest already lists @stdlib/blas/base/xerbla as a dependency under task: 'benchmark'.

Actual Results

Undefined symbols for architecture arm64:
  "_xerbla", referenced from:
      _dger in ccCfvnCt.o
ld: symbol(s) not found for architecture arm64
collect2: error: ld returned 1 exit status
make: *** [benchmark.length.out] Error 1

Linker fails with an unresolved reference to xerbla. The dependency's .f source is never surfaced because library-manifest only declares Fortran sources under task: 'build'.

Version

develop

Environments

N/A

Browser Version

No response

Node.js / npm Version

No response

Platform

Reproduces on any platform with gfortran + make (linux/macOS verified).

Checklist

• Read and understood the Code of Conduct.
• Searched for existing issues and pull requests.

[officiallyanee]

Hi, I think a possible solution for this would be to include the fortran source files in the source of task source config and filter them out in C script for resolving source file (like this line of compile_c_benchmark), and similarly to update compile_fortran_benchmark & the Makefiles in benchmark to be more general (without hardcoded paths) following C's example with fortran compiler.
Since C & Fortran benchmarks use their source files directly (that is .c & .f respectively) it shouldn't cause an issue.

[batpigandme]

Thanks @officiallyanee, the script-parity observation is the right one — compile_fortran_benchmark should pull src/libraries/libpath from the manifest the way compile_c_benchmark does, and the per-package Fortran Makefiles can drop their hardcoded SOURCE_FILES ?= ../../src/<pkg>.f lines once it does.

One thing to fold in: per @kgryte's framing at office hours last week, I think the language-aware bit should ideally live in @stdlib/utils/library-manifest rather than in the wrappers.

I took a look at it to see what the lang-aware change would touch. Ideally the change would low-impact shape that doesn't require a manifest.json schema change. One thought was that we could add lang as an options filter (third arg to manifest()), applied post-resolve as an extension-based partition over src. Three files:

• lib/defaults.js: add 'lang': '' to the defaults.
• lib/validate.js: validate as a string with allowed values (e.g. 'c' | 'fortran' | '').
• lib/main.js: apply the filter at the end of the final loop, after dedupe and before path-convention conversion.

Callers then look like:

manifest( fpath, { 'task': 'benchmark', ... }, { 'lang': 'c', ... } ).src;       // .c only
manifest( fpath, { 'task': 'benchmark', ... }, { 'lang': 'fortran', ... } ).src; // .f only
manifest( fpath, { 'task': 'benchmark', ... }, opts ).src;                       // unfiltered (default)

So compile_fortran_benchmark passes { 'lang': 'fortran' }, compile_c_benchmark passes { 'lang': 'c' }, and any future Fortran-linking context that goes through library-manifest gets the right partition for free. Plus, empty-string default = no filter, so existing call sites are unaffected.

If I understand correctly, if we were to do lang as a conditions key (like task, os, blas), every manifest.json would need a lang entry in options and per-language confs.

I could be missing something in the blast radius, though—so definitely take a peek for yourself.

Worth flagging that the manifest edits will need to land on every Fortran-bearing transitive dep (xerbla and friends), not just the leaf packages whose benchmarks happen to be broken today, but I think that's already what you have in mind.

Want to take a stab at this one? No rush. I know everyone's got exams!

[batpigandme]

Reconsidering this in the context of last week's office hours: I got the scope wrong in my last comment. The lang post-filter idea is at the wrong layer, and I want to correct that before anyone starts building on it.

The Fortran missing-symbol thing is actually one symptom of a bigger structural mismatch in how library-manifest resolves dependencies.

Right now when library-manifest resolves task: 'benchmark' for a top-level package, it propagates task: 'benchmark' recursively to every dep in the tree. That works by accident — deps tend to have a task: 'benchmark' conf with a reasonable src list, so things compile. It breaks when a dep doesn't have native benchmarks at all (I think float32 base assert packages were mentioned last week), and it's why Fortran symbols go missing: the dep traversal isn't expressing "what files do I contribute when I'm pulled in as someone else's dependency?" — it's just reusing each dep's own benchmark conf as a stand-in.

The lang filter idea would have been filtering an already-incomplete src list. The list itself isn't being built correctly when crossing dep boundaries — filtering it differently doesn't help.

Revised thoughts on a structural fix:

1. Add an explicit "as-dependency" configuration to every C/Fortran package — separate from task: 'build' | 'benchmark' | 'examples'. This declares what files/includes/libraries to merge in when the package is consumed as a dep, regardless of the top-level task.
2. Update the recursive dep resolution in library-manifest (specifically the dep-merge loop in lib/main.js, ~lines 161–204) so it uses that as-dependency conf instead of propagating the top-level task straight through.

Once dep-confs explicitly list their Fortran contributions (.f files), the missing-symbol case goes away. Per @kgryte's description last week, the Fortran resolution issue is general — it applies to native add-ons, examples, source implementations, and benchmarks — so it has to be fixed at this layer, not in any one compile script.

What this means for the thread: the manifest-level change is bigger than a single-file (or few-file) patch — it touches every C/Fortran package's manifest.json (hundreds of files), the library-manifest recursion, backward-compat considerations, and schema docs. Probably needs its own issue with maintainer scoping before anyone takes a stab.

That said, the compile_fortran_benchmark parity work you identified, @officiallyanee, is still real and still actionable on its own — the wrapper is underpowered relative to compile_c_benchmark, and fixing that doesn't depend on the manifest restructuring. That's still a helpful patch if you want to focus there.

I could still be wrong about the blast radius — definitely take a peek at the dep-merge loop in lib/main.js yourself. And still no rush on any of it — exam season is real 🙂.

[officiallyanee]

Thanks for the pointers!
Some ref for as-dependency: Athan's comment on missing task config, the issues raised for the same: 1 & 2.
I will add the patch for the parity work as soon as I can! And would look into the blast radius too.

[batpigandme]

Some notes from @kgryte's observations in today's office hours on the broader library-manifest refactor direction (we can iterate and revise, etc., just didn't want to lose the info):

Two coupled fronts:

• Configuration inheritance for manifest.json. ESLint-style presets — you define named base configs, and other configs can extend/inherit from them, only specifying overrides. This addresses the repetition problem in packages with lots of permutations (especially BLAS with hardware-accelerated bindings). Named configs may live outside the confs array entirely, and multiple inheritance is on the table.
• "Dependency mode." What's currently missing from every manifest.json is a conf that says "when this package is used as a dependency, here's the minimal set of files/headers/libraries that are necessary." That's why the current dep resolution works by accident (propagating the top-level task down the tree) and breaks when a dep doesn't have a matching task conf. The dependency-mode conf would use the inheritance mechanism from the first front.

Implementation plan per Athan:

• Incubate: add inheritance support to @stdlib/utils/library-manifest
• Prototype: test on one or two packages of varying complexity to prove it out
• Automate: develop a script to convert the existing ~thousands of binding files to the new format

Mara's postulation
What this means for the Fortran issue specifically: the missing-symbol bug falls out of the dependency-mode conf — once deps explicitly declare their Fortran contributions (.f files) in a dep-mode conf, the resolver picks them up correctly.

I think the compile_fortran_benchmark wrapper parity issue remains (it only resolves include from the manifest, while compile_c_benchmark resolves include, src, libraries, and libpath), since doesn't depend on the inheritance work. So, that could probably be its own little patch.

[officiallyanee]

Some more refs that might be useful: Eslint's old extends key & Eslint's new ideology for flat cascade
The former seems more useful in terms of design that will be required here, except I think one concern would be the merging of dependencies, given that we have different base for all different combination of os, blas, wasm we support.
(These do differ in build and non-build in blas pkgs though, for src files & hence their dependencies)

[officiallyanee]

Also a note on compile_fortran_benchmark: achieving parity depends on having fortran source files resolved through manifest.json, which currently lacks them. Because if we were to go ahead then the defaults set for SOURCE_FILES in pkg-level Makefiles for benchmarks would be override by c source files in benchmark conf which would cause issues.
However, LIBRARIES, LIBPATH, os and BLAS awareness can be brought to parity without depending on the manifest changes.

[officiallyanee]

Benchmarks for fortran are run under pre-push hook but not in CI. 😮