Module test_support

Module test_support 

Source
Expand description

Runtime support for #[ktstr_test] integration tests.

Provides the registration type, distributed slice, VM launcher, and result evaluation. Includes guest-side profraw flush for coverage-instrumented builds.

The entry point for test authors is the crate::ktstr_test attribute macro; see the user-facing Writing Tests guide shipped with the crate’s mdbook for end-to-end examples and the full attribute grammar.

§Consumer API

Test authors interact primarily with the #[ktstr_test] proc macro; programmatic test generation can instead populate KtstrTestEntry values into the KTSTR_TESTS linkme distributed slice. The remaining items in this module are runtime glue invoked by the macro-generated code and the ktstr / cargo-ktstr binaries.

§Module layout

Implementation is split across 19 production submodules re-exported at test_support::* for a flat public API: args (CLI argument extraction), dispatch (ktstr / cargo-ktstr CLI entry points), entry (scheduler + test-entry types), entry_validate (KtstrTestEntry::validate + phase helpers split out of entry.rs), eval (host-side VM result evaluation), host_class (shared host-insufficiency error classification), metrics (payload stdout → Metric list), output (guest-output and console parsing), payload (Payload / MetricCheck / Metric / Polarity), probe (auto-repro and BPF probe pipeline), probe_metrics (host-side BPF map introspection), profraw (coverage flush), runtime (pub mod — neutral home for verbose/shm-size/config-file-parts shared by eval and probe so they don’t circularly depend on each other), shell_descriptor (wire-format struct shared between the test binary’s --ktstr-shell-test=<NAME> producer and cargo-ktstr’s shell-mode consumer), wprof (#[cfg(feature = "wprof")] — Perfetto-trace wire constants + .wprof.pb shape assertions), sidecar (per-run JSON records), staged (pub(crate) mod — staged-payload writer), timefmt (ISO-8601

  • run-id helpers), and topo (topology override parsing).

A #[cfg(test)] pub(crate) mod test_helpers exists for cross-file test wiring; it is not part of the production surface.

Re-exports§

pub use wprof::PERFETTO_TRACE_PACKETS_TAG;
pub use wprof::WPROF_PB_MIN_BYTES;
pub use wprof::assert_wprof_pb_shape;
pub use entry::KTSTR_SCHEDULERS;
pub use entry::KTSTR_TESTS;

Modules§

runtime
Runtime configuration primitives shared by eval and probe.
wprof
Public wire-format constants + assertion helpers for the wprof Perfetto-trace artifacts produced by #[ktstr_test(wprof, ...)] tests.

Structs§

BpfMapWrite
Host-side BPF map write performed during VM execution.
CgroupPath
Validated cgroup parent path.
KernelUnavailable
Marker error for “the test harness can’t find a kernel image to boot the VM against”. Wraps the actionable diagnostic that resolve_test_kernel emits when neither KTSTR_TEST_KERNEL nor any standard cache / sysroot location produced a bootable image.
KtstrTestEntry
Registration entry for an #[ktstr_test]-annotated function.
MemSideCache
Re-exports of topology types for use in KtstrTestEntry statics generated by the #[ktstr_test] macro. HMAT Type 2 memory-side cache descriptor.
Metric
A single extracted metric from a payload’s output.
MetricHint
Payload-author metric declaration: polarity + display unit.
NumaDistance
Re-exports of topology types for use in KtstrTestEntry statics generated by the #[ktstr_test] macro. NxN inter-NUMA-node distance matrix.
NumaNode
Re-exports of topology types for use in KtstrTestEntry statics generated by the #[ktstr_test] macro. Per-NUMA-node configuration.
Payload
A test payload — either a scheduler or a userspace binary to run inside the guest VM.
PayloadMetrics
All metrics extracted from a single payload run plus the process exit code.
PerfDeltaAssertion
A per-test performance-regression assertion that cargo ktstr perf-delta --noise-adjust enforces when this test is compared across two commits — and that a normal cargo ktstr test run IGNORES.
PerfDeltaAssertionRecord
Owned, serialized mirror of crate::test_support::PerfDeltaAssertion. The public declaration type uses &'static str (so it stays const/E0493-safe on the entry) and therefore cannot Deserialize into an owned value; this String-backed record is the sidecar carrier the perf-delta compare reads. pub because it is a field of the pub SidecarResult (constructed across the workspace, including by the cargo-ktstr binary crate); the author-facing declaration type is crate::test_support::PerfDeltaAssertion.
Scheduler
Definition of a scheduler for the test framework.
SchedulerArtifactRequirement
One exact scheduler executable requirement reported by a warmed test binary.
SchedulerJson
JSON shape projected from a registered Scheduler. Each entry carries the complete execution identity needed to compare declarations: scheduler name, a BinaryKindJson-tagged binary specification (Discover / Path / Eevdf / KernelBuiltin), guest setup, static scheduler configuration, per-scheduler default TopologyJson, always-on scheduler args, declared kernel set, and gauntlet constraints. Assertion overrides and inline per-test config-file content are intentionally omitted.
SchedulerListEntry
A SchedulerJson plus the number of #[ktstr_test]s which use it as either their primary or a staged scheduler. Emitted (as a JSON array) by the --ktstr-list-schedulers probe so cargo ktstr affected can enumerate declared schedulers AND skip those with zero test uses when producing its CI matrix, in one probe.
SchedulerManifestProbe
Complete scheduler metadata projected from one test binary.
SchedulerTestJson
A single #[ktstr_test] paired with one scheduler it exercises. A test contributes one entry for its primary scheduler and one for every staged scheduler. Included in SchedulerManifestProbe::tests and also emitted as the legacy JSON array for --ktstr-list-scheduler-tests.
ShellTestDescriptor
Wire-format descriptor exchanged between a test binary and cargo ktstr shell --test <NAME> to let the shell VM mirror the named #[ktstr_test]’s topology, scheduler, wprof config, and performance mode.
SidecarResult
Test result sidecar written to KTSTR_SIDECAR_DIR for post-run analysis.
Sysctl
A key=value sysctl applied to the guest before the scheduler starts, injected into the guest kernel cmdline as sysctl.<key>=<value>.
SysctlJson
Ordered JSON representation of one Sysctl.
Topology
Re-exports of topology types for use in KtstrTestEntry statics generated by the #[ktstr_test] macro. CPU topology specification with NUMA memory topology.
TopologyConstraints
Gauntlet topology filtering constraints.
TopologyConstraintsJson
JSON-friendly mirror of TopologyConstraints — the host-side Option<u32> fields serialize as null (default serde behavior; no skip_serializing_if) rather than the Some(N)/None-tagged shapes serde uses for Option inside larger struct graphs. Field semantics match TopologyConstraints verbatim; see that type for per-field documentation.
TopologyJson
JSON-friendly mirror of Topology for the verifier wire format. Captures the four-tuple shape (numa nodes × LLCs × cores × threads) the per-scheduler baseline topology was declared with. The verifier uses this when computing the sweep matrix — the baseline anchors the default cell when no gauntlet preset matches the scheduler’s constraints.
WatchBpfMap
Host-side, observer-effect-free read of a NAMED scheduler BPF-map field, surfaced as an assertable run-level metric.

Enums§

BinaryKindJson
JSON-friendly form of SchedulerSpec tagged so the verifier dispatch can exhaustively match on the variant. Discover and Path both carry a string identifier; Eevdf and KernelBuiltin both signal “no BPF to verify”.
BpfMapAgg
Aggregation for a WatchBpfMap target.
HostClass
Outcome of classifying a test-body error against the host-insufficiency taxonomy.
MetricCheck
Assertion check evaluated against an extracted PayloadMetrics (or the exit code for MetricCheck::ExitCodeEq).
MetricStream
Which of the payload’s output streams a Metric was extracted from.
OutputFormat
How the framework extracts metrics from a payload’s output.
PayloadKind
How a payload is launched inside the guest.
Polarity
Regression direction for a metric.
ResolveSource
Provenance of a scheduler binary returned by resolve_scheduler.
SchedulerKind
Discriminator for the test’s scheduler-spec shape on the --ktstr-shell-test=<NAME> wire format. Wire-byte-compatible with the prior stringly-typed "eevdf" | "discover" | "path" | "kernel_builtin" values via #[serde(rename_all = "snake_case")]; the typed enum replaces the stringly-typed boundary so a rename on either side (producer in dispatch::maybe_dispatch_shell_test, consumer in cargo_ktstr::misc::shell) is a compile error instead of a silent banner-emit-gate regression.
SchedulerSpec
How to specify the scheduler for an #[ktstr_test].
ThreadLookup
Outcome of scanning the flat metric list for a tid-keyed thread entry. Distinguishes “tid not present” from “tid present but allocated_bytes missing” AND from “probe emitted more than MAX_SCAN_INDEX contiguous threads without the caller’s tid appearing in the prefix” — so a caller can issue a precise diagnostic instead of a blanket “not found”.
ThroughputDenomination
Which time base denominates this sidecar’s workload-THROUGHPUT rate metrics (taobench_*_ops_per_cpu_sec / schbench_*loops_per_cpu_sec / iteration_rate, and any future throughput rate). Some rate key names predate the denomination change, so a number from a wall-era sidecar is NOT comparable to a CPU-era sidecar — the compare pipeline keys its row pairing and its group-averaging on this marker so cross-denomination values are never silently folded or diffed (see PairingKey::from_row and the denomination_mismatches counter in CompareReport).

Constants§

DEFAULT_HOST_CGROUP_PARENT
Default cgroup parent path for host_only tests when KTSTR_HOST_CGROUP_PARENT is unset. Suitable for both root (writable directly) and non-root (operator pre-creates /sys/fs/cgroup/ktstr with appropriate ownership, OR overrides via KTSTR_HOST_CGROUP_PARENT to point at a path inside a delegated subtree) invocations. See resolve_host_cgroup_parent for the env-override path and build_host_cgroup_manager for the cgroup-v2 Mode B/C delegation wire-up.
EXIT_FAIL
Process exit code for a Fail verdict (or any expect_err satisfaction failure).
EXIT_INCONCLUSIVE
Process exit code for an Inconclusive verdict (a zero-denominator ratio gate that could not evaluate).
EXIT_PASS
Process exit code for a Pass verdict (and for the Skip path, which degenerates to Pass because the test never ran).
MAX_SCAN_INDEX
Safety bound on the snapshots.*.threads.N.tid scan in lookup_thread, snapshot_worker_allocated, thread_count, and snapshot_count. Realistic probe runs see at most a few dozen threads in a single-allocator worker process; hitting this cap indicates either an unexpectedly wide target or a flat-metric schema change that broke the terminator convention.
MAX_WALK_DEPTH
Hard cap on recursion depth in walk. Object and array children past this depth are skipped and a single [tracing::warn!] fires. Serde_json’s default parser recursion limit is 128, so this caps us well below that; a hand-built serde_json::Value that bypasses the parser can still reach arbitrary depth, so an explicit walker guard is the last line of defence against a stack overflow.
WALK_TRUNCATION_SENTINEL_NAME
Sentinel metric name emitted when walk hits MAX_WALK_DEPTH and skips a subtree. Callers of walk_json_leaves / extract_metrics that want to distinguish “no deep metrics present” from “deep metrics dropped by the depth cap” scan the returned Vec<Metric> for a metric whose name equals this constant — its value is the depth at which truncation occurred, so nested failures at different subtrees produce one sentinel per trigger.

Statics§

KTSTR_SCHEDULERS
Distributed slice collecting all declare_scheduler! registrations via linkme. Each entry is a &'static Scheduler pointing at a const emitted by the macro. Runtime lookup and retained private probe commands walk this slice; cargo ktstr discovery reads the parallel versioned ELF-stamp registry without starting the binary.
KTSTR_TESTS
Distributed slice collecting all #[ktstr_test] entries via linkme.

Functions§

analyze_sidecars
Collect sidecar JSON files and return the full gauntlet analysis.
capture_starvation_witness
Witnessed-starvation evidence for capture-bearing assertions: the host dilation D that PROVES the run was contended enough to explain missing/degraded periodic captures, or None on a quiet (or unwitnessed) host.
classify_host_error
Classify a test-body error against the host-insufficiency taxonomy.
collect_pool
Pool every sidecar JSON under every run directory at root.
count_indexed_metrics
Walk 0..cap applying key_fn(i) to form a metric name and count how many consecutive indices yield a present metric. Stops at the first miss — the probe’s walk_json_leaves flattening yields indices 0..N contiguously, so the first gap is the array terminator. Returns the count, which may be cap if every index below the bound is present (inconclusive — the caller should treat cap as “saturated scan, real count may be larger”).
default_post_vm_periodic_fired
Default post_vm callback emitted by #[ktstr_test] when the attribute omits post_vm = .... Asserts that at least one periodic snapshot produced REAL BPF state during the workload window WHEN periodic was configured (num_snapshots > 0); when periodic was disabled (num_snapshots == 0), the helper is a no-op and returns Ok.
extract_metrics
Extract metrics from a payload’s captured output per its declared OutputFormat.
find_metric
Find a metric by exact name. Returns None if absent.
find_metric_u64
Fetch a metric by exact name and return its numeric value as a u64. Returns None if the metric is absent. Thin wrapper around find_metric + value as u64 for the common numeric-lookup shape.
find_scheduler
Look up a registered scheduler by its Scheduler::name field (the name = "..." value supplied to declare_scheduler!, not the SCREAMING_SNAKE_CASE const identifier the macro emits). Returns None if no registered scheduler matches.
find_test
Look up a registered test function by name.
flat_metrics_dump
Flatten the full (name, value) pair list for diagnostic rendering inside error messages. Returned as an owned Vec<(&str, f64)> so call sites spell the diagnostic as a single {:?} formatter argument instead of re-typing the .iter().map(...).collect() chain at every site.
format_run_artifact_footer
Render the cargo ktstr test post-run footer: for each run directory written at or after since, name every FAILED test and the concrete path to each of its artifacts (failure dump, auto-repro dump, stats sidecar, wprof trace), plus a per-dir count of stats sidecars and wprof traces.
has_metric
Does the flat metric list contain a metric with this exact name? Thin wrapper around find_metric for the common existence check — avoids forcing every call site to spell .is_some().
host_capacity
Host capacity triple (cpus, llcs, max_cpus_per_llc) used to filter gauntlet topology presets against what the host can actually schedule. Both dispatch::list_tests_* (gauntlet variant filter) and dispatch::list_verifier_cells_all (verifier sweep filter) share this single source of truth so the two filters never drift. Reads available_parallelism() for CPU count + HostTopology::from_sysfs() for LLC layout; falls back to single-LLC + single-cpu-per-llc when sysfs is unavailable.
is_truncation_sentinel_name
Predicate for “this metric name / map key is the walk-truncation sentinel.” Centralises the literal-equality check so every consumer stays in sync when the sentinel name changes, and so future sentinel variants (e.g. a parser-rejection sentinel) can be threaded through one predicate instead of scattered string literals.
ktstr_main
Nextest protocol handler.
lookup_thread
Extract the allocated_bytes / deallocated_bytes values for worker_tid from snapshot 0 in the flat metric list produced by walk_json_leaves over the probe’s JSON output.
newest_run_dir
Find the most recently modified run directory under runs_root.
periodic_starvation_gate
Periodic-capture starvation gate for capture-requiring post_vm assertions: starved_below_minimum_skip keyed on the bridge’s REAL (non-placeholder) capture count vs the test’s stated minimum. Call at the TOP of a post_vm callback with the same minimum its assertions enforce: periodic_starvation_gate(result, 2)?;. Tests whose assertions key on a narrower data-bearing count (e.g. “N captures carrying per-cgroup PSI”) should ALSO call starved_below_minimum_skip with that count at the assertion site. No-op when the entry configured no periodic captures.
post_vm_skip
Request a test SKIP from a post_vm / post_vm_unconditional callback: return Err(post_vm_skip(reason)) when the run is INCONCLUSIVE — the VM could not produce the artifact the assertion needs (e.g. a load-starved VM whose BPF probe never attached, leaving a placeholder failure dump), as distinct from a real regression. The framework detects the attached HostSkipRequest marker and converts the run to crate::assert::AssertResult::skip instead of a failure.
profraw_target_dir
Resolve the llvm-cov-target directory for profraw output.
resolve_host_cgroup_parent
Resolve the cgroup parent path for host_only tests.
resolve_scheduler
Resolve a scheduler binary from a SchedulerSpec.
resolve_test_kernel
Find a kernel image for running tests.
run_ktstr_test
Host-side entry point: build a VM, boot it with --ktstr-test-fn=NAME, extract profraw from SHM, and return the test result.
runs_root
Resolve the parent directory that holds all test-run subdirectories.
sanitize_kernel_label
Sanitise a kernel label (the producer-side identity emitted by cargo ktstr’s resolver) into a nextest-safe identifier of the shape kernel_[a-z0-9_]+.
sidecar_dir
Resolve the sidecar output directory for the current test process.
snapshot_count
Count the number of snapshots.N.timestamp_unix_sec entries in the flat metric list, capped at MAX_SCAN_INDEX.
snapshot_worker_allocated
Extract snapshots.{snap_idx}.threads[*].allocated_bytes for the thread whose tid matches worker_tid. Returns ThreadLookup so callers distinguish “tid absent” from “cap hit before tid seen” from “allocated_bytes sibling missing” — parallel to lookup_thread, which covers the single-snapshot path.
stall_ejection_skip
Environmental skip for a zero-scheduler-activity reading (e.g. nr_dispatched / nr_yielded read 0 across every periodic sample) that the guest’s OWN sched_ext runnable-stall watchdog explains: when the console carries a watchdog ScxExitKind::Stall exit AND capture_starvation_witness proves the host was contended, the BPF scheduler was ejected because a descheduling host dilated the 5s GUEST-time stall window past the timeout — not because the scheduler failed to dispatch. Returns Err(post_vm_skip(..)) in that case; Ok otherwise, so a Stall on a quiet host (no witness) or any non-Stall zero still reaches the caller’s own regression ensure!.
starved_below_minimum_skip
Generic sub-minimum starvation skip: Err(post_vm_skip(..)) when the run produced FEWER data-bearing units than the caller’s assertion requires (have < need) AND capture_starvation_witness proves the host was contended enough to explain the shortfall; Ok(()) otherwise. The caller passes the SAME count its own assertion checks (real captures, PSI-carrying captures, series length, …) so the gate and the assertion cannot disagree about the boundary — an earlier zero-only gate let 1 of 6 runs through to fail on a >= 2 minimum under the identical environmental condition. Quiet-host shortfalls (zero OR sub-minimum) still fall through and fail with the caller’s specific diagnosis.
thread_count
Count the number of snapshots.0.threads.N.{Ok,Err}.tid entries in the flat metric list, capped at MAX_SCAN_INDEX. Each index carries exactly one variant wrapper (Ok or Err); the count terminates at the first index where neither exists.
walk_json_leaves
Walk numeric leaves of a JSON value, emitting Metrics keyed by dotted paths.

Type Aliases§

PostVmCallback
Shared callback signature for the KtstrTestEntry::post_vm and KtstrTestEntry::post_vm_unconditional host-side hooks. Both fields wrap this same shape in Option<_>; the alias collapses the open-coded fn(&crate::vmm::VmResult) -> anyhow::Result<()> repetition at the field declarations and at the matching with_post_vm{,_unconditional} builder parameters. Future post-VM hooks (e.g. an expect_auto_repro artifact-existence checker) plug into the same shape without triplicating the signature.