NUMA Aware Work Stealing

[Bathtor]

When running with pinned threads, prioritise NUMA local work-stealing (significantly) over stealing from further away cores.

Status

• Partially available with "manual" specification of a PU distance matrix (from Partial NUMA Aware Workstealing #10 )
• Automatic discovery is blocked on better hwloc support or some topology alternative

[Bathtor]

Check https://crates.io/crates/hwloc to get topology information

[Bathtor]

C&P from Slack for reference:

lkroll 14:52
Alright, I’ve been looking into the NUMA aware workstealing and it’s a bit tricky. The only way I found to get some sensible topology information is via rust-hwloc(2), which is a FFI wrapper around libhwloc. So far so good, but the people who wrote the wrapper clearly have no idea how Rust FFIs are supposed to work and their interfaces suck balls. In particular some of their methods throw SIGSEGV because they try to dereference NULL pointers within an unsafe block (which isn’t exposed…the function itself is written as if it were safe).
14:52
Now, we could try to fix this…but that would probably mean we’ll end up maintaining rust-hwloc in long term (at least that is my experience with these kind of semi-abandoned projects)
14:53
And when I say ‘we’, I obviously mean ‘you’, since I’ll be leaving after December :D
14:53
How would you like me to proceed on this?^^
New
Max 14:56
I see, well it isn't critical to have right now anyways. More of a wish. Thanks for checking, I might have a look into it again in the future 😄