Quick tests fails with ASRL enabled on CentOS

[racardoso]

This problem only affects CentOS (possible Rhel). Ubuntu runs fine. Tested on CentOS7.

HHVM Version

HHVM next (ppc64el)
HHVM master (ppc64el)
HHVM upstream/master (ppc64el)
HHVM 3.15.1 (ppc64el)

Tested on Release and Debug mode.

Standalone code, or other way to reproduce the problem

Compile and install HHVM on CentOS7.

After installation run quick tests

cd hphp/test
./run quick

Expected result

1105/1105 tests passed as in Ubuntu

Actual result

1065 tests failed with a segmentation fault (SIGSEGV) or a assert fail when run in Debug mode:

Assertion failure: /home/hhvm-3.15.0/hphp/runtime/vm/jit/fixup.cpp:108: void HPHP::jit::FixupMap::fixupWork(HPHP::ExecutionContext*, HPHP::ActRec*) const: assertion `nextRbp && nextRbp != rbp && "Missing fixup for native call"' failed.

[racardoso]

Run a failed test at GDB got:

$ gdb --args ../hhvm/hhvm -c hhvm/hphp/test/config.ini -vEval.EnableArgsInBacktraces=true -vEval.EnableIntrinsicsExtension=true -vRepo.Local.Mode=-- -vRepo.Central.Path=hhvm/hphp/hhvm/verify.hhbc -vEval.Jit=true -vResourceLimit.CoreFileSize=0 --file 'quick/BindM.php'
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-80.el7
(gdb) run
Starting program: hphp/test/../hhvm/hhvm -c /hphp/test/config.ini -vEval.EnableArgsInBacktraces=true -vEval.EnableIntrinsicsExtension=true -vRepo.Local.Mode=-- -vRepo.Central.Path=hphp/hhvm/verify.hhbc -vEval.Jit=true -vResourceLimit.CoreFileSize=0 --file quick/BindM.php
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/power8/libthread_db.so.1".
array(1) {
  [0]=>
  &string(2) "hi"
}
array(1) {
  [0]=>
  &string(2) "hi"
}
[Inferior 1 (process 606) exited normally]

Running this test out of gdb can reproduce the error.

Assertion failure: /home/hhvm-3.15.0/hphp/runtime/vm/jit/fixup.cpp:108: void HPHP::jit::FixupMap::fixupWork(HPHP::ExecutionContext*, HPHP::ActRec*) const: assertion `nextRbp && nextRbp != rbp && "Missing fixup for native call"' failed

[racardoso]

Enable address randomization on GDB
set disable-randomization off
Reproduces the problem. GDB disable virtual address randomization since it can cause problems when debug code.

(gdb) set disable-randomization off
(gdb) run
Starting program: hhvm/hphp/test/../hhvm/hhvm -c hphp/test/config.ini -vEval.EnableArgsInBacktraces=true -vEval.EnableIntrinsicsExtension=true -vRepo.Local.Mode=-- -vRepo.Central.Path=hhvm/hphp/hhvm/verify.hhbc -vEval.Jit=true -vResourceLimit.CoreFileSize=0 --file quick/BindM.php
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/power8/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x0000010003200028 in ?? ()

[racardoso]

Address 0x0000010003200028 points to a li, 1 instruction which makes no sense since this instruction do not access memory and should not throw a SIGSEGV. Checking memory permissions appears that the block where the jitted instructions are write is not marked with exec permission hence we got a segmentation fault when try to execute the instruction due a permission fault.

Program received signal SIGSEGV, Segmentation fault.
0x0000010003200028 in ?? ()
(gdb) call getpid()
$1 = 610
(gdb) !pmap -x 610
610:   hhvm/hphp/test/../hhvm/hhvm -c hphp/test/config.ini -vEval.EnableArgsInBacktraces=true -vEval.EnableIntrinsicsExtension=true -vRepo.Local.Mode=-- -vRepo.Central.Path=/hhvm/hphp/hhvm/verify.hhbc -vEval.Jit=true -vResourceLimit.CoreFileSize=0 --file quick/BindM.php
Address           Kbytes     RSS   Dirty Mode  Mapping
0000000010000000   64704   27904     128 r-x-- hhvm
0000000013f30000     320     320     320 r---- hhvm
0000000013f80000     960     320     320 rw--- hhvm
0000000014070000     576     384     384 rw---   [ anon ]
0000010003150000  105152     256     256 rw---   [ anon ]
 

[racardoso]

There's two workarounds for this problem:

1. Disable Address Space Randomization (ASRL)
   Edit /etc/sysctl.conf and add:

kernel.randomize_va_space = 0

then run

sysctl -p

2. Disable selinux

Disable selinux works too:
Edit the file /etc/sysconfig/selinux and change SELINUX variable to disabled

I still need to confirm if this bug affects only PPC64 machines or affects X86. I still not sure if this is a HHVM or a CentOS bug.