Add size-gated hugepage optimization for large allocations#291
Add size-gated hugepage optimization for large allocations#291hsane-dev wants to merge 1 commit intogoogle:masterfrom
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
c562ce0 to
d24787f
Compare
This patch adds optional hugepage optimization by applying madvise(MADV_HUGEPAGE) to allocations that are large enough to benefit from transparent hugepages. Current behavior: - Uses MADV_NOHUGEPAGE for infrequent access patterns, MADV_HUGEPAGE only for memory collapsing - No proactive hugepages for large allocations Enhanced behavior: - Adds MADV_HUGEPAGE for allocations >= kHugePageSize (2MB) - Only applies to allocations that can actually benefit from hugepages - Avoids kernel overhead on small allocations The size threshold ensures we only hint hugepage usage for allocations that can actually be backed by hugepages, avoiding unnecessary kernel overhead for smaller allocations.
d24787f to
f0bf91a
Compare
|
I could imagine this still causes regressions for some people -- eg if you aren't expecting hugepages and use an arena allocator with a 2MiB allocation size, now you'll get a full 2M RSS whereas before you got granular 4k page allocation as your arena filled up? |
|
I think I'm comfortable with someone getting a hugepage: TCMalloc likes hugepages, and manages memory in a hugepage-aware way. We're assuming they're there, even if they aren't, when making allocation decisions. That said, I'm not sure the size restriction buys us much. Allocations to the system allocator are typically making 1GB-sized allocations, and we end up faulting in the memory later when it |
Summary
Adds proactive hugepage optimization to improve memory performance for allocations ≥2MB while preserving existing memory management behavior.
Changes
MmapRegion::Alloc()insystem_allocator.hto applyMADV_HUGEPAGEfor large allocations>= kHugePageSize) to avoid overhead on small allocationsMADV_NOHUGEPAGElogic for infrequent access patterns