Running Tests
Every #[ktstr_test] boots a fresh KVM microVM with the topology the
test declares, on the exact kernel you target. cargo ktstr test
resolves that kernel (building and caching it when needed) and wraps
cargo nextest run, so nextest’s filtering, retries, and parallelism
all apply.
Read the assertion, timeline, scheduler log, monitor verdict, dumps, and replay command.
Expand one test across presets, then select variants with nextest filters or a CI budget.
Use run sidecars and perf-delta instead of comparing terminal output by eye.
Quick reference
# Run all tests
cargo ktstr test --kernel ../linux
# Run a specific test
cargo ktstr test --kernel ../linux -- -E 'test(sched_basic_proportional)'
# Run all ktstr-managed tests, skipping non-ktstr tests in the same crate
cargo ktstr test --kernel ../linux -- -E 'test(/^ktstr/)'
# Run ignored gauntlet variants
cargo ktstr test --kernel ../linux -- --run-ignored ignored-only -E 'test(gauntlet/)'
More patterns:
| Goal | Command |
|---|---|
| Re-run one exact ktstr case | cargo ktstr test --kernel 7.0 -- -E 'test(=ktstr/my_test)' |
| Compare two kernels | cargo ktstr test --kernel 6.14 --kernel 7.0 |
| Stay inside a CI time budget | KTSTR_BUDGET_SECS=300 cargo ktstr test --kernel 7.0 |
| Open the same VM shape manually | cargo ktstr shell --kernel 7.0 --test my_test |
What’s in this chapter
- cargo ktstr — the host-side command:
kernel resolution, test dispatch, replay, coverage, export, plus the
interactive shell,
topo,ctprof, andlocksdebugging tools (and the standalonektstrbinary that carries them without cargo). - Gauntlet — run every test across a matrix of topology presets.
- BPF Verifier Sweep — verify, attach, and dispatch every declared scheduler across topologies.
- Reading Failure Output — what a failed test prints, section by section, and how to investigate.
- Auto-Repro — the second VM that replays a scheduler crash with probes attached.
- Runs and Regression Gates — result
sidecars,
stats, andperf-delta.
Test names and variants
Tests registered through #[ktstr_test] show up in nextest output
under these name shapes:
ktstr/{name}— single-kernel VM run.host/{name}— ahost_onlytest. It never boots a VM and never multiplies across kernels.ktstr/{name}/{kernel}— one case per (test × kernel) when--kernelresolves to two or more kernels.gauntlet/{name}/{preset}— one case per topology preset (see Gauntlet).gauntlet/{name}/{preset}/{kernel}— the full (test × preset × kernel) expansion under a multi-kernel run.
This is what those names look like in a real run:
Nextest run ID 98581174-246f-4824-a170-50992df166d7 with nextest profile: default
Starting 1 test across 121 binaries (12531 tests skipped)
PASS [ 34.459s] (1/1) ktstr::failure_dump_e2e ktstr/failure_dump_renders_bss_fields
... ktstr::worktype_coverage_fork_gauntlet_e2e gauntlet/worktype_fork_gauntlet_covers_all_arms/4cpu-2llc-nosmt ktstr::worktype_coverage_fork_gauntlet_e2e gauntlet/worktype_fork_gauntlet_covers_all_arms/8cpu-2llc-smt ktstr::worktype_coverage_fork_gauntlet_e2e gauntlet/worktype_fork_gauntlet_covers_all_arms/9cpu-3llc-nosmt
Filter by prefix with -E 'test(/^ktstr/)', -E 'test(/^host/)', or
-E 'test(/^gauntlet/)'.
The distinct host/ prefix is also an admission identity. cargo ktstr
keeps ordinary host tests CPU-bounded while allowing all cheap VM-resource
waiters to reach ktstr’s topology scheduler. Older ktstr releases listed
host-only entries under the VM namespace; full-name filters for those entries
should use host/ now.
Tip
test(NAME)is a substring match; the exact-match formtest(=NAME)matches the full nextest name, prefix included. Usetest(=ktstr/sched_basic_proportional), not the bare function name —test(=sched_basic_proportional)matches nothing.
The {kernel} suffix is a sanitized kernel label: kernel_ prefix,
lowercase, non-alphanumeric characters collapsed to _ — 6.16.1
becomes kernel_6_16_1, and a path spec becomes
kernel_path_{basename}_{hash6} (with _dirty appended when the
source tree has uncommitted changes). The 6-character hash
disambiguates two source paths that share a basename.
RUST_BACKTRACE=1 controls panic backtraces and verbose failure
output, not guest console streaming — see
Reading Failure Output for the
investigation knobs.
Budget-based test selection
Set KTSTR_BUDGET_SECS to select the subset of tests that maximizes
configuration coverage within a time budget — useful for CI pipelines
and quick smoke tests:
KTSTR_BUDGET_SECS=300 cargo ktstr test --kernel ../linux
The selector encodes each test as a bitset of properties (scheduler, topology class, SMT, workload characteristics) and greedily picks the tests with the highest marginal coverage per estimated second, with duration estimates accounting for VM boot overhead by vCPU count. A summary is printed to stderr during budget-mode listing:
ktstr budget: 42/1200 tests, 295/300s used, 38/38 configurations covered
Testing your own scheduler
Declare it with declare_scheduler! and reference it from
#[ktstr_test(scheduler = ...)] — see
Scheduler Definitions and
the Test a New Scheduler recipe.