Removing ort/ndarray compatibility pin also breaks Linux arm64 builds

[chenrui333]

The compatibility pin from #1312 appears to be needed for more than just the original x86_64 Homebrew case.

In Homebrew we currently carry the patch from that PR in the magika formula:

• rust: pin ort/ndarray for x86_64 build compatibility #1312

When I tried removing that patch in this Homebrew PR:

• magika: drop intel support Homebrew/homebrew-core#272963

Linux arm64 also failed to build from source in CI.

Direct failing Linux arm job:

• https://github.com/Homebrew/homebrew-core/actions/runs/23243289467/job/67565534579?pr=272963

The useful part of the failing linker output on ubuntu-22.04-arm was:

/usr/bin/ld: ...libort_sys-....rlib(...): undefined reference to `__cxa_call_terminate`
/usr/bin/ld: ...inference_session.cc.o: undefined reference to `__isoc23_strtol`
/usr/bin/ld: ...allocator_adapters.cc.o: undefined reference to `__isoc23_strtoll`
/usr/bin/ld: ...parser.cc.o: undefined reference to `__isoc23_strtoull`
collect2: error: ld returned 1 exit status

This was coming from the ort / ort-sys objects, not Magika's own Rust code.

For comparison, rebuilding current main in Homebrew with the patch still present succeeded on Linux arm64:

• https://github.com/Homebrew/homebrew-core/actions/runs/23245392373

That run produced a successful ubuntu-22.04-arm bottle, while the patch-removal PR above did not.

So from the Homebrew side, dropping the pin regresses Linux arm64 in addition to the earlier x86_64 compatibility issue.

Questions:

• Is Linux arm64 / aarch64-unknown-linux-gnu intended to be supported for the Rust CLI right now?
• If yes, should the fix from rust: pin ort/ndarray for x86_64 build compatibility #1312 (or a follow-up equivalent) stay in upstream until ort / ndarray can be moved forward safely?
• If not, it would help to document that current Linux support is narrower than generic linux.

If useful, I can also provide the exact Homebrew formula patch block and the full failing CI excerpt from the patch-removal PR.

[ia0]

Thanks for the issue!

• Is Linux arm64 / aarch64-unknown-linux-gnu intended to be supported for the Rust CLI right now?

Yes. This is one of the target we release in GitHub: https://github.com/google/magika/releases/tag/cli%2Fv1.0.2

• If yes, should the fix from rust: pin ort/ndarray for x86_64 build compatibility #1312 (or a follow-up equivalent) stay in upstream until ort / ndarray can be moved forward safely?

I need to investigate what's going on. Maybe there's a discrepancy between our workflows and those of Homebrew, or something broke recently.

[ia0]

I can't reproduce on CI so this is a problem from the Homebrew build environment.

[ia0]

@chenrui333 did it ever work for ARM64 Linux on Homebrew? I made a comment on the PR that could help.

[chenrui333]

yeah, it worked before.

[ia0]

I see, so it could be a GCC version mismatch.

[chenrui333]

Quick clarification from the Homebrew side after looking at the PR discussion and your temporary test PR #1325:

The failing Homebrew PR only does two things:

• removes the existing compatibility patch from rust: pin ort/ndarray for x86_64 build compatibility #1312
• adds depends_on arch: :arm64

It does not otherwise change the toolchain.

In the failing Linux arm64 job, Cargo is already linking -lstdc++, and Homebrew reports HOMEBREW_CXX=g++-12, so this does not look like a simple “missing C++ standard library” case.

The unresolved symbols are coming from the downloaded ort-sys / ONNX Runtime objects, for example:

• __cxa_call_terminate
• __isoc23_strtol
• __isoc23_strtoll
• __isoc23_strtoull

So from our side this looks more like an ABI mismatch in the downloaded ort binary than a generic compiler-package problem.

The important A/B result is:

• current Homebrew main with the rust: pin ort/ndarray for x86_64 build compatibility #1312 patch still present built arm64_linux successfully
• the patch-removal PR failed on arm64_linux

So the compatibility pin from #1312 appears to be shielding Linux arm64 as well, not just the earlier x86_64 Homebrew case.

If you want, I can also paste the exact linker command / excerpt from the failing Homebrew arm64 job here.

[ia0]

• current Homebrew main with the rust: pin ort/ndarray for x86_64 build compatibility #1312 patch still present built arm64_linux successfully

Yes, it's because the patch uses an older ort which uses an older ONNX Runtime which uses an older toolchain. To build the current version, GCC 14 is needed.

Homebrew reports HOMEBREW_CXX=g++-12

Ok that's the issue. Is it possible to bump this?

[luengnat]

Is it related to https://lists.llvm.org/pipermail/libcxx-commits/2024-June/085127.html ?