[Go] Improve 32-bit support (`GOARCH=386`)

[joellubi]

Describe the enhancement requested

There are several parts of the code that currently fail on 32-bit systems like 386.

1. One problem is that the refCount int64 we currently use for reference-counted resources requires manual alignment on 32-bit systems when used with atomic.AddInt64. This was raised in [Go] parquet/pqarrow: panic: unaligned 64-bit atomic operation arrow#43186.
2. Another problem specific to 386 arch is that certain functions with archictecture-specific implementations do not have variants defined for 386, causing them to panic when compiled for that system.

apache/arrow#43206 starts to address these issues for a narrow subset of the codebase. This issue is intended to track the remaining work needed to add full support across the codebase.

Component(s)

Go

[jas4711]

Debian supports three 32-bit platforms: i386, armhf, and armel. So we would like to see arrow-go work on 32-bit platforms. On i386, our stripped down build works fine, but there are self-tests failures that only occur on i386 and not amd64:

https://salsa.debian.org/jas/golang-github-apache-arrow-go/-/jobs/7074351

Build log with >1000 FAIL's:

https://salsa.debian.org/jas/golang-github-apache-arrow-go/-/jobs/7074351/artifacts/raw/debian/output/golang-github-apache-arrow-go_18.1.0-1+salsaci+20250211+5_i386.build

For example the first failure:

=== NAME  TestBitmapOps/TestBitmapAnd/aligned/left_offset_3/right_offset_3/out_offset_3
    panic.go:262: test panicked: runtime error: invalid memory address or nil pointer dereference
        goroutine 192 [running]:
        runtime/debug.Stack()
        	/usr/lib/go-1.23/src/runtime/debug/stack.go:26 +0x83
        github.com/stretchr/testify/suite.failOnPanic(0xa8ff408, {0x843d980, 0x8769920})
        	/builds/jas/golang-github-apache-arrow-go/debian/output/source_dir/_build/src/github.com/stretchr/testify/suite/suite.go:89 +0x38
        github.com/stretchr/testify/suite.recoverAndFailOnPanic(0xa8ff408)
        	/builds/jas/golang-github-apache-arrow-go/debian/output/source_dir/_build/src/github.com/stretchr/testify/suite/suite.go:83 +0x3b
        panic({0x843d980, 0x8769920})
        	/usr/lib/go-1.23/src/runtime/panic.go:785 +0x103
        github.com/apache/arrow-go/v18/arrow/bitutil.alignedBitmapOp({0x84a0cc8, 0x84a0ccc, 0x0}, {0xaae59c4, 0x3, 0x3}, {0xaae5a04, 0x3, 0x3}, 0x3, ...)
        	/builds/jas/golang-github-apache-arrow-go/debian/output/source_dir/_build/src/github.com/apache/arrow-go/v18/arrow/bitutil/bitmaps.go:516 +0x624
        github.com/apache/arrow-go/v18/arrow/bitutil.BitmapOp({0x84a0cc8, 0x84a0ccc, 0x0}, {0xaae59c4, 0x3, 0x3}, {0xaae5a04, 0x3, 0x3}, 0x3, ...)
        	/builds/jas/golang-github-apache-arrow-go/debian/output/source_dir/_build/src/github.com/apache/arrow-go/v18/arrow/bitutil/bitmaps.go:545 +0x163
        github.com/apache/arrow-go/v18/arrow/bitutil.BitmapOpAlloc({0x850dde4, 0x8781c20}, {0x84a0cc8, 0x84a0ccc, 0x0}, {0xaae59c4, 0x3, 0x3}, {0xaae5a04, 0x3, ...}, ...)
        	/builds/jas/golang-github-apache-arrow-go/debian/output/source_dir/_build/src/github.com/apache/arrow-go/v18/arrow/bitutil/bitmaps.go:555 +0x16f
        github.com/apache/arrow-go/v18/arrow/bitutil.BitmapAndAlloc({0x850dde4, 0x8781c20}, {0xaae59c4, 0x3, 0x3}, {0xaae5a04, 0x3, 0x3}, 0x3, 0x3, ...)
        	/builds/jas/golang-github-apache-arrow-go/debian/output/source_dir/_build/src/github.com/apache/arrow-go/v18/arrow/bitutil/bitmaps.go:568 +0xca
        github.com/apache/arrow-go/v18/arrow/bitutil_test.(*BitmapOpSuite).testAligned.func1.1.1()
        	/builds/jas/golang-github-apache-arrow-go/debian/output/source_dir/_build/src/github.com/apache/arrow-go/v18/arrow/bitutil/bitmaps_test.go:411 +0x28a
        github.com/stretchr/testify/suite.(*Suite).Run.func1(0xa8ff408)
        	/builds/jas/golang-github-apache-arrow-go/debian/output/source_dir/_build/src/github.com/stretchr/testify/suite/suite.go:115 +0x141
        testing.tRunner(0xa8ff408, 0xa87a200)
        	/usr/lib/go-1.23/src/testing/testing.go:1690 +0x119
        created by testing.(*T).Run in goroutine 191
        	/usr/lib/go-1.23/src/testing/testing.go:1743 +0x3d1
...
--- FAIL: TestBitmapOps (1.05s)
    --- FAIL: TestBitmapOps/TestBitmapAnd (0.51s)
        --- FAIL: TestBitmapOps/TestBitmapAnd/aligned (0.03s)
...
            --- FAIL: TestBitmapOps/TestBitmapAnd/aligned/left_offset_3 (0.01s)
                --- FAIL: TestBitmapOps/TestBitmapAnd/aligned/left_offset_3/right_offset_3 (0.00s)
                    --- FAIL: TestBitmapOps/TestBitmapAnd/aligned/left_offset_3/right_offset_3/out_offset_3 (0.00s)
                        --- PASS: TestBitmapOps/TestBitmapAnd/aligned/left_offset_3/right_offset_3/out_offset_3/zero-length (0.00s)
                    --- FAIL: TestBitmapOps/TestBitmapAnd/aligned/left_offset_3/right_offset_3/out_offset_19 (0.00s)
                        --- PASS: TestBitmapOps/TestBitmapAnd/aligned/left_offset_3/right_offset_3/out_offset_19/zero-length (0.00s)
                    --- FAIL: TestBitmapOps/TestBitmapAnd/aligned/left_offset_3/right_offset_3/out_offset_27 (0.00s)
                        --- PASS: TestBitmapOps/TestBitmapAnd/aligned/left_offset_3/right_offset_3/out_offset_27/zero-length (0.00s

Any ideas?

[zeroshade]

I must have made a mistake on some of the build constraints. Let me take a look and do some testing for 32-bit arch and get back to you.

[sfc-gh-jmichalak]

Hi,

Such errors can still happen on 32-bit platforms (reference), even after bumping Arrow to v18.

The errors no longer appear after upgrading to a 64-bit platform.

[zeroshade]

That would likely be fixed by #326

[Cacsjep]

whats the status here, because I have armv7hf devices (cctv cameras) where I now got panic: unaligned 64-bit atomic operation because of using influxdb3 client that use underhood arrow-go? I would really need a fix here :( could i help ?

[zeroshade]

Did the influxdb3 client update to v18.3.0? Are you still experiencing the panic on v18.3.0?

[Cacsjep]

No, they don't update yet, I create an issue, to request and update to latest. They use v15

[zeroshade]

Any updates here? Is this reproducible with 18.3.1?

[zeroshade]

The 32-bit architectural issues identified here have been successfully addressed by PR #277 and PR #326. Closing this issue!