Module test_support

Expand description

Runtime support for #[ktstr_test] integration tests.

Provides the registration type, distributed slice, VM launcher, and result evaluation. Includes guest-side profraw flush for coverage-instrumented builds.

The entry point for test authors is the crate::ktstr_test attribute macro; see the user-facing Writing Tests guide shipped with the crate’s mdbook for end-to-end examples and the full attribute grammar.

§Consumer API

Test authors interact primarily with the #[ktstr_test] proc macro; programmatic test generation can instead populate KtstrTestEntry values into the KTSTR_TESTS linkme distributed slice. The remaining items in this module are runtime glue invoked by the macro-generated code and the ktstr / cargo-ktstr binaries.

§Module layout

Implementation is split across 19 production submodules re-exported at test_support::* for a flat public API: args (CLI argument extraction), dispatch (ktstr / cargo-ktstr CLI entry points), entry (scheduler + test-entry types), entry_validate (KtstrTestEntry::validate + phase helpers split out of entry.rs), eval (host-side VM result evaluation), host_class (shared host-insufficiency error classification), metrics (payload stdout → Metric list), output (guest-output and console parsing), payload (Payload / MetricCheck / Metric / Polarity), probe (auto-repro and BPF probe pipeline), probe_metrics (host-side BPF map introspection), profraw (coverage flush), runtime (pub mod — neutral home for verbose/shm-size/config-file-parts shared by eval and probe so they don’t circularly depend on each other), shell_descriptor (wire-format struct shared between the test binary’s --ktstr-shell-test=<NAME> producer and cargo-ktstr’s shell-mode consumer), wprof (#[cfg(feature = "wprof")] — Perfetto-trace wire constants + .wprof.pb shape assertions), sidecar (per-run JSON records), staged (pub(crate) mod — staged-payload writer), timefmt (ISO-8601

run-id helpers), and topo (topology override parsing).

A #[cfg(test)] pub(crate) mod test_helpers exists for cross-file test wiring; it is not part of the production surface.

Re-exports§

pub use wprof::PERFETTO_TRACE_PACKETS_TAG;
pub use wprof::WPROF_PB_MIN_BYTES;
pub use wprof::assert_wprof_pb_shape;
pub use entry::KTSTR_SCHEDULERS;
pub use entry::KTSTR_TESTS;

Modules§

runtime: Runtime configuration primitives shared by eval and probe.
wprof: Public wire-format constants + assertion helpers for the wprof Perfetto-trace artifacts produced by #[ktstr_test(wprof, ...)] tests.

Structs§

BpfMapWrite: Host-side BPF map write performed during VM execution.
CgroupPath: Validated cgroup parent path.
KernelUnavailable: Marker error for “the test harness can’t find a kernel image to boot the VM against”. Wraps the actionable diagnostic that resolve_test_kernel emits when neither KTSTR_TEST_KERNEL nor any standard cache / sysroot location produced a bootable image.
KtstrTestEntry: Registration entry for an #[ktstr_test]-annotated function.
MemSideCache: Re-exports of topology types for use in KtstrTestEntry statics generated by the #[ktstr_test] macro. HMAT Type 2 memory-side cache descriptor.
Metric: A single extracted metric from a payload’s output.
MetricHint: Payload-author metric declaration: polarity + display unit.
NumaDistance: Re-exports of topology types for use in KtstrTestEntry statics generated by the #[ktstr_test] macro. NxN inter-NUMA-node distance matrix.
NumaNode: Re-exports of topology types for use in KtstrTestEntry statics generated by the #[ktstr_test] macro. Per-NUMA-node configuration.
Payload: A test payload — either a scheduler or a userspace binary to run inside the guest VM.
PayloadMetrics: All metrics extracted from a single payload run plus the process exit code.
PerfDeltaAssertion: A per-test performance-regression assertion that cargo ktstr perf-delta --noise-adjust enforces when this test is compared across two commits — and that a normal cargo ktstr test run IGNORES.
PerfDeltaAssertionRecord: Owned, serialized mirror of crate::test_support::PerfDeltaAssertion. The public declaration type uses &'static str (so it stays const/E0493-safe on the entry) and therefore cannot Deserialize into an owned value; this String-backed record is the sidecar carrier the perf-delta compare reads. pub because it is a field of the pub SidecarResult (constructed across the workspace, including by the cargo-ktstr binary crate); the author-facing declaration type is crate::test_support::PerfDeltaAssertion.
Scheduler: Definition of a scheduler for the test framework.
SchedulerJson: JSON shape projected from a registered Scheduler. Each entry carries scheduler name, a BinaryKindJson-tagged binary specification (Discover / Path / Eevdf / KernelBuiltin), per-scheduler default TopologyJson, always-on scheduler args, declared kernel set, and gauntlet constraints. Internal fields (assertion overrides, sysctls, kargs, cgroup parent, config-file plumbing) are intentionally omitted.
SchedulerListEntry: A SchedulerJson plus the number of #[ktstr_test]s declared against it. Emitted (as a JSON array) by the --ktstr-list-schedulers probe so cargo ktstr affected can enumerate declared schedulers AND skip those with zero tests when producing its CI matrix, in one probe.
SchedulerTestJson: A single #[ktstr_test] paired with its declared scheduler’s name. Emitted (as a JSON array) by the --ktstr-list-scheduler-tests probe so --relevant can map each test to its scheduler and select the tests whose scheduler a diff affects.
ShellTestDescriptor: Wire-format descriptor exchanged between a test binary and cargo ktstr shell --test <NAME> to let the shell VM mirror the named #[ktstr_test]’s topology, scheduler, wprof config, and performance mode.
SidecarResult: Test result sidecar written to KTSTR_SIDECAR_DIR for post-run analysis.
Sysctl: A key=value sysctl applied to the guest before the scheduler starts, injected into the guest kernel cmdline as sysctl.<key>=<value>.
Topology: Re-exports of topology types for use in KtstrTestEntry statics generated by the #[ktstr_test] macro. CPU topology specification with NUMA memory topology.
TopologyConstraints: Gauntlet topology filtering constraints.
TopologyConstraintsJson: JSON-friendly mirror of TopologyConstraints — the host-side Option<u32> fields serialize as null (default serde behavior; no skip_serializing_if) rather than the Some(N)/None-tagged shapes serde uses for Option inside larger struct graphs. Field semantics match TopologyConstraints verbatim; see that type for per-field documentation.
TopologyJson: JSON-friendly mirror of Topology for the verifier wire format. Captures the four-tuple shape (numa nodes × LLCs × cores × threads) the per-scheduler baseline topology was declared with. The verifier uses this when computing the sweep matrix — the baseline anchors the default cell when no gauntlet preset matches the scheduler’s constraints.
WatchBpfMap: Host-side, observer-effect-free read of a NAMED scheduler BPF-map field, surfaced as an assertable run-level metric.

Enums§

BinaryKindJson: JSON-friendly form of SchedulerSpec tagged so the verifier dispatch can exhaustively match on the variant. Discover and Path both carry a string identifier; Eevdf and KernelBuiltin both signal “no BPF to verify”.
BpfMapAgg: Aggregation for a WatchBpfMap target.
HostClass: Outcome of classifying a test-body error against the host-insufficiency taxonomy.
MetricCheck: Assertion check evaluated against an extracted PayloadMetrics (or the exit code for MetricCheck::ExitCodeEq).
MetricStream: Which of the payload’s output streams a Metric was extracted from.
OutputFormat: How the framework extracts metrics from a payload’s output.
PayloadKind: How a payload is launched inside the guest.
Polarity: Regression direction for a metric.
ResolveSource: Provenance of a scheduler binary returned by resolve_scheduler.
SchedulerKind: Discriminator for the test’s scheduler-spec shape on the --ktstr-shell-test=<NAME> wire format. Wire-byte-compatible with the prior stringly-typed "eevdf" | "discover" | "path" | "kernel_builtin" values via #[serde(rename_all = "snake_case")]; the typed enum replaces the stringly-typed boundary so a rename on either side (producer in dispatch::maybe_dispatch_shell_test, consumer in cargo_ktstr::misc::shell) is a compile error instead of a silent banner-emit-gate regression.
SchedulerSpec: How to specify the scheduler for an #[ktstr_test].
ThreadLookup: Outcome of scanning the flat metric list for a tid-keyed thread entry. Distinguishes “tid not present” from “tid present but allocated_bytes missing” AND from “probe emitted more than MAX_SCAN_INDEX contiguous threads without the caller’s tid appearing in the prefix” — so a caller can issue a precise diagnostic instead of a blanket “not found”.

Constants§

DEFAULT_HOST_CGROUP_PARENT: Default cgroup parent path for host_only tests when KTSTR_HOST_CGROUP_PARENT is unset. Suitable for both root (writable directly) and non-root (operator pre-creates /sys/fs/cgroup/ktstr with appropriate ownership, OR overrides via KTSTR_HOST_CGROUP_PARENT to point at a path inside a delegated subtree) invocations. See resolve_host_cgroup_parent for the env-override path and build_host_cgroup_manager for the cgroup-v2 Mode B/C delegation wire-up.
EXIT_FAIL: Process exit code for a Fail verdict (or any expect_err satisfaction failure).
EXIT_INCONCLUSIVE: Process exit code for an Inconclusive verdict (a zero-denominator ratio gate that could not evaluate).
EXIT_PASS: Process exit code for a Pass verdict (and for the Skip path, which degenerates to Pass because the test never ran).
MAX_SCAN_INDEX: Safety bound on the snapshots.*.threads.N.tid scan in lookup_thread, snapshot_worker_allocated, thread_count, and snapshot_count. Realistic probe runs see at most a few dozen threads in a single-allocator worker process; hitting this cap indicates either an unexpectedly wide target or a flat-metric schema change that broke the terminator convention.
MAX_WALK_DEPTH: Hard cap on recursion depth in walk. Object and array children past this depth are skipped and a single [tracing::warn!] fires. Serde_json’s default parser recursion limit is 128, so this caps us well below that; a hand-built serde_json::Value that bypasses the parser can still reach arbitrary depth, so an explicit walker guard is the last line of defence against a stack overflow.
WALK_TRUNCATION_SENTINEL_NAME: Sentinel metric name emitted when walk hits MAX_WALK_DEPTH and skips a subtree. Callers of walk_json_leaves / extract_metrics that want to distinguish “no deep metrics present” from “deep metrics dropped by the depth cap” scan the returned Vec<Metric> for a metric whose name equals this constant — its value is the depth at which truncation occurred, so nested failures at different subtrees produce one sentinel per trigger.

Statics§

KTSTR_SCHEDULERS: Distributed slice collecting all declare_scheduler! registrations via linkme. Each entry is a &'static Scheduler pointing at a const emitted by the macro. The verifier discovers schedulers by spawning the test binary with --ktstr-list-schedulers; a per-binary ctor walks this slice and serializes each entry to JSON.
KTSTR_TESTS: Distributed slice collecting all #[ktstr_test] entries via linkme.

Functions§

analyze_sidecars: Collect sidecar JSON files and return the full gauntlet analysis.
classify_host_error: Classify a test-body error against the host-insufficiency taxonomy.
collect_pool: Pool every sidecar JSON under every run directory at root.
count_indexed_metrics: Walk 0..cap applying key_fn(i) to form a metric name and count how many consecutive indices yield a present metric. Stops at the first miss — the probe’s walk_json_leaves flattening yields indices 0..N contiguously, so the first gap is the array terminator. Returns the count, which may be cap if every index below the bound is present (inconclusive — the caller should treat cap as “saturated scan, real count may be larger”).
default_post_vm_periodic_fired: Default post_vm callback emitted by #[ktstr_test] when the attribute omits post_vm = .... Asserts that at least one periodic snapshot produced REAL BPF state during the workload window WHEN periodic was configured (num_snapshots > 0); when periodic was disabled (num_snapshots == 0), the helper is a no-op and returns Ok.
extract_metrics: Extract metrics from a payload’s captured output per its declared OutputFormat.
find_metric: Find a metric by exact name. Returns None if absent.
find_metric_u64: Fetch a metric by exact name and return its numeric value as a u64. Returns None if the metric is absent. Thin wrapper around find_metric + value as u64 for the common numeric-lookup shape.
find_scheduler: Look up a registered scheduler by its Scheduler::name field (the name = "..." value supplied to declare_scheduler!, not the SCREAMING_SNAKE_CASE const identifier the macro emits). Returns None if no registered scheduler matches.
find_test: Look up a registered test function by name.
flat_metrics_dump: Flatten the full (name, value) pair list for diagnostic rendering inside error messages. Returned as an owned Vec<(&str, f64)> so call sites spell the diagnostic as a single {:?} formatter argument instead of re-typing the .iter().map(...).collect() chain at every site.
format_run_artifact_footer: Render the cargo ktstr test post-run footer: for each run directory written at or after since, name every FAILED test and the concrete path to each of its artifacts (failure dump, auto-repro dump, stats sidecar, wprof trace), plus a per-dir count of stats sidecars and wprof traces.
has_metric: Does the flat metric list contain a metric with this exact name? Thin wrapper around find_metric for the common existence check — avoids forcing every call site to spell .is_some().
host_capacity: Host capacity triple (cpus, llcs, max_cpus_per_llc) used to filter gauntlet topology presets against what the host can actually schedule. Both dispatch::list_tests_* (gauntlet variant filter) and dispatch::list_verifier_cells_all (verifier sweep filter) share this single source of truth so the two filters never drift. Reads available_parallelism() for CPU count + HostTopology::from_sysfs() for LLC layout; falls back to single-LLC + single-cpu-per-llc when sysfs is unavailable.
is_truncation_sentinel_name: Predicate for “this metric name / map key is the walk-truncation sentinel.” Centralises the literal-equality check so every consumer stays in sync when the sentinel name changes, and so future sentinel variants (e.g. a parser-rejection sentinel) can be threaded through one predicate instead of scattered string literals.
ktstr_main: Nextest protocol handler.
lookup_thread: Extract the allocated_bytes / deallocated_bytes values for worker_tid from snapshot 0 in the flat metric list produced by walk_json_leaves over the probe’s JSON output.
newest_run_dir: Find the most recently modified run directory under runs_root.
post_vm_skip: Request a test SKIP from a post_vm / post_vm_unconditional callback: return Err(post_vm_skip(reason)) when the run is INCONCLUSIVE — the VM could not produce the artifact the assertion needs (e.g. a load-starved VM whose BPF probe never attached, leaving a placeholder failure dump), as distinct from a real regression. The framework detects the attached HostSkipRequest marker and converts the run to crate::assert::AssertResult::skip instead of a failure.
profraw_target_dir: Resolve the llvm-cov-target directory for profraw output.
resolve_host_cgroup_parent: Resolve the cgroup parent path for host_only tests.
resolve_scheduler: Resolve a scheduler binary from a SchedulerSpec.
resolve_test_kernel: Find a kernel image for running tests.
run_ktstr_test: Host-side entry point: build a VM, boot it with --ktstr-test-fn=NAME, extract profraw from SHM, and return the test result.
runs_root: Resolve the parent directory that holds all test-run subdirectories.
sanitize_kernel_label: Sanitise a kernel label (the producer-side identity emitted by cargo ktstr’s resolver) into a nextest-safe identifier of the shape kernel_[a-z0-9_]+.
sidecar_dir: Resolve the sidecar output directory for the current test process.
snapshot_count: Count the number of snapshots.N.timestamp_unix_sec entries in the flat metric list, capped at MAX_SCAN_INDEX.
snapshot_worker_allocated: Extract snapshots.{snap_idx}.threads[*].allocated_bytes for the thread whose tid matches worker_tid. Returns ThreadLookup so callers distinguish “tid absent” from “cap hit before tid seen” from “allocated_bytes sibling missing” — parallel to lookup_thread, which covers the single-snapshot path.
thread_count: Count the number of snapshots.0.threads.N.{Ok,Err}.tid entries in the flat metric list, capped at MAX_SCAN_INDEX. Each index carries exactly one variant wrapper (Ok or Err); the count terminates at the first index where neither exists.
walk_json_leaves: Walk numeric leaves of a JSON value, emitting Metrics keyed by dotted paths.

Type Aliases§

PostVmCallback: Shared callback signature for the KtstrTestEntry::post_vm and KtstrTestEntry::post_vm_unconditional host-side hooks. Both fields wrap this same shape in Option<_>; the alias collapses the open-coded fn(&crate::vmm::VmResult) -> anyhow::Result<()> repetition at the field declarations and at the matching with_post_vm{,_unconditional} builder parameters. Future post-VM hooks (e.g. an expect_auto_repro artifact-existence checker) plug into the same shape without triplicating the signature.