Module host_heap

Module host_heap 

Source
Expand description

Heap-state snapshot for the running ktstr binary.

HostHeapState is a thin snapshot of the process’s jemalloc allocator state at capture time: active / allocated / resident / mapped bytes plus the arena count. It rides along inside HostContext::heap_state so a sidecar reader can correlate scheduler-test outcomes with the ktstr tool’s own memory footprint — e.g. distinguish a legitimate regression from one where the runner itself OOM-pressured the host.

§jemalloc is always linked

tikv-jemalloc-ctl declares a non-optional dependency on tikv-jemalloc-sys, which builds and links libjemalloc unconditionally. So even consumers that do NOT install tikv_jemallocator::Jemalloc as #[global_allocator] carry libjemalloc in their binary, and every mallctl call from this module resolves to libjemalloc’s implementation rather than a libc stub. mallctl reads succeed regardless of which allocator #[global_allocator] resolves to.

What differs is the meaning of the numbers:

  • When jemalloc IS #[global_allocator] (every shipped binary in this workspace, via the central #[global_allocator] in src/lib.rs gated on the cli-bins feature), every heap allocation flows through jemalloc and stats.allocated / stats.active report real application usage in the tens-to-hundreds of MiB range.
  • When jemalloc is linked but is NOT #[global_allocator] (downstream consumers using ktstr as a library without opting into jemallocator), jemalloc still initializes its arenas but the application never allocates through it. stats.allocated and stats.active return Some(0) in that case. arenas.narenas is still populated (jemalloc computes it as 4 * ncpus at init time) and stats.resident / stats.mapped reflect jemalloc’s own metadata footprint — small but non-zero.

collect collapses the “jemalloc linked but unused” shape (allocated_bytes == Some(0) && active_bytes == Some(0)) to None at the HostContext::heap_state call site, so sidecars from non-jemallocator consumers do not carry misleading mostly-zero rows. The jemalloc-used signal (non-zero allocated AND active) is what warrants sidecar space.

§stats feature is required for stats reads

libjemalloc only tracks stats.* counters when the C library is built with --enable-stats; without it the mallctl reads still succeed but return zero. The stats feature on both tikv-jemalloc-ctl and tikv-jemallocator in Cargo.toml forces the C build flag — host_heap::collect depends on this. arenas.narenas is independent of --enable-stats; it reports correctly either way.

§Epoch discipline

stats.* reads return cached values; the cache refreshes when the epoch mallctl is advanced. collect advances the epoch exactly once before issuing reads so each snapshot reflects post-advance state. Callers that invoke collect back-to-back see fresh reads every time because each call advances the epoch again.

Side effect of epoch::advance(): libjemalloc flushes per-thread stat caches into the shared counters under a mallctl-internal lock. The operation is thread-safe per jemalloc’s mallctl contract — concurrent collect calls from multiple ktstr threads are defined-behavior (though pointless; each caller sees its own refreshed snapshot and the last writer wins in the caches).

Structs§

HostHeapState
Heap-state snapshot for the running process’s jemalloc allocator.

Functions§

collect
Capture the running process’s jemalloc heap state.