Module host_heap

Module host_heap 

Source
Expand description

Heap-state snapshot for the running ktstr binary.

HostHeapState is a thin snapshot of the process’s jemalloc allocator state at capture time: active / allocated / resident / mapped bytes plus the arena count. It rides along inside HostContext::heap_state so a sidecar reader can correlate scheduler-test outcomes with the ktstr tool’s own memory footprint — e.g. distinguish a legitimate regression from one where the runner itself OOM-pressured the host.

§jemalloc is always linked

tikv-jemalloc-ctl declares a non-optional dependency on tikv-jemalloc-sys, which builds and links libjemalloc unconditionally. So even consumers that do NOT install tikv_jemallocator::Jemalloc as #[global_allocator] carry libjemalloc in their binary, and every mallctl call from this module resolves to libjemalloc’s implementation rather than a libc stub. mallctl reads succeed regardless of which allocator #[global_allocator] resolves to.

What differs is the meaning of the numbers:

  • When jemalloc IS #[global_allocator] (every shipped binary in this workspace declares it at its binary crate root), every heap allocation flows through jemalloc and stats.allocated / stats.active report real application usage in the tens-to-hundreds of MiB range.
  • When jemalloc is linked but is NOT #[global_allocator] (downstream consumers using ktstr as a library without opting into jemallocator), the application never allocates through it. Calling mallctl still initializes jemalloc and can give its own small metadata allocations non-zero stats.allocated / stats.active values, so counter magnitude is not a sound way to identify the process allocator.

Shipped binaries declare jemalloc at their crate roots and mark that fact once at process entry. Host-context capture checks the marker before calling collect, so allocator-agnostic integration-test binaries neither initialize jemalloc nor publish its irrelevant metadata as application heap usage.

§stats feature is required for stats reads

libjemalloc only tracks stats.* counters when the C library is built with --enable-stats; without it the mallctl reads still succeed but return zero. The stats feature on both tikv-jemalloc-ctl and tikv-jemallocator in Cargo.toml forces the C build flag — host_heap::collect depends on this. arenas.narenas is independent of --enable-stats; it reports correctly either way.

§Epoch discipline

stats.* reads return cached values; the cache refreshes when the epoch mallctl is advanced. collect advances the epoch exactly once before issuing reads so each snapshot reflects post-advance state. Callers that invoke collect back-to-back see fresh reads every time because each call advances the epoch again.

Side effect of epoch::advance(): libjemalloc flushes per-thread stat caches into the shared counters under a mallctl-internal lock. The operation is thread-safe per jemalloc’s mallctl contract — concurrent collect calls from multiple ktstr threads are defined-behavior (though pointless; each caller sees its own refreshed snapshot and the last writer wins in the caches).

Structs§

HostHeapState
Heap-state snapshot for the running process’s jemalloc allocator.

Functions§

collect
Capture the running process’s jemalloc heap state.