./libc.so.6.instr: ELF load command address/offset not page-aligned

[B03901108]

Hi, when I tested Cipherfix on the openssl/ecdh and openssl/ecdsa examples, I encountered the following errors. ./app.instr: error while loading shared libraries: ./libc.so.6.instr: ELF load command address/offset not page-aligned. I ran the tests on an Ubuntu 22.04.2 LTS sever (processor: Intel Xeon E5-1650 v3). I used OpenSSL 3.0.2, Intel Pin 3.26, dotnet 6.0.113, and NASM 2.15.05.

Take openssl/ecdh as an example. I first ran under cipherfix directory
./analyze.sh path_to_cipherfix/examples/openssl/ecdh /usr/lib/ssl "1;4;5" app 10 perf
./instrument.sh path_to_cipherfix/examples/openssl/ecdh enhanced xsprng.

Based on the "Candidates" list in instrument.sh results, I then appended
Mm 0000557302c37260 (<app+00001260> malloc.plt)
Mm 0000557302c37280 (<app+00001280> CRYPTO_malloc.plt)
Mm 00007fa62579f380 (<libc.so.6+00028380> malloc)
Mm 00007fa625cca300 (<libcrypto.so.3+001b7300> CRYPTO_malloc)
Mr 00007fa625cca730 (<libcrypto.so.3+001b7730> CRYPTO_realloc)
to structure.out and re-ran
./instrument.sh path_to_cipherfix/examples/openssl/ecdh enhanced xsprng.

instrument.sh gave the following messages.
Running static instrumentation
path_to_cipherfix/static-instrumentation path_to_cipherfix
path_to_cipherfix/static-instrumentation/StaticInstrumentation path_to_cipherfix/static-instrumentation
Allocated fixed RNG vector registers:
State: XMM14
Key: XMM15
Help: XMM13
Checking image app...
Checking image ld-linux-x86-64.so.2...
Dynamic linker, skipping...
Checking image [vdso]...
vDSO, skipping...
Checking image libcrypto.so.3...
Checking image libc.so.6...
Instrumenting image app as app.instr...
Instrumenting image libcrypto.so.3 as libcrypto.so.3.instr...
Skipping empty instrumented basic block #19b879
Skipping empty instrumented basic block #1acfec
Skipping empty instrumented basic block #1ad064
Instrumenting image libc.so.6 as libc.so.6.instr...
Instrumentation completed.
path_to_cipherfix/static-instrumentation
path_to_cipherfix
Instrumentation completed

Under the instr-enhanced-xsprng directory, I ran chmod +x ./app.instr and ./app.instr 10. Then I got the error. I tried different instrumentation settings (e.g., instr-base-aesrng), but the error remains. I tried the other example (openssl/ecdsa) with "enhanced xsprng" & "enhanced evalmarker" and still encountered the errors.

[JanWichelmann]

Hi, thank you for interest in Cipherfix, and the detailed bug report!

Unfortunately, due to the high complexity of ELF rewriting and our instrumentation approach, our proof-of-concept implementation is a bit unstable on other systems. It works best on AMD EPYC processors and Ubuntu 20.04. Also, the pre-built Docker images should provide a solid basis.

We also encountered the error in question when testing on Ubuntu 22.04, and it is most likely caused by libc.so.6 having a slightly different structure than on other systems - more specifically, the first few ELF sections seem to be a bit denser / have less space in between, which breaks a heuristic in our ELF rewriter. I haven't been able to come up with a fix for that, yet.

As a workaround, you could try to downgrade to Ubuntu 20.04, rebuild libc or use our Docker images. However, there may be other errors due to unsupported instructions on Xeon processors.

[B03901108]

Great thanks for the detailed reply. I am now trying out older versions of Ubuntu. Could the code work on AMD EPYC 9654? (It seems that AMD EPYC 9654 still uses SEV-SNP, but I am unsure if the ciphertext side channels still exist.)

[JanWichelmann]

The code may work on your CPU, but Zen 4 supports more instructions than Zen 3 (most notably, AVX-512), so the instrumentation may fail in some cases.