cargo ktstr
cargo ktstr is the host-side command for the whole workflow: it
resolves (and if needed builds and caches) the kernel, then drives
cargo nextest run so every #[ktstr_test] boots its VM against
exactly the kernel you asked for.
test, replay, and common flags for ordinary ktstr suites.
kernel, shell, export, and cache-management flows.
verifier, stats, perf-delta, ctprof, and CI targeting.
Install
cargo install --locked ktstr # installs both `ktstr` and `cargo-ktstr`
The test-fixture binaries (jemalloc probes, schbench/taobench
validators) are behind the non-default integration feature and are
not installed by default. To build from a workspace checkout
instead: cargo build --bin cargo-ktstr.
Task map
| I want to… | Command | Depth |
|---|---|---|
| Run tests on a kernel | cargo ktstr test | this page |
| Re-run last session’s failures | cargo ktstr replay | this page |
| Manage cached kernels | cargo ktstr kernel | this page |
| Sweep schedulers through the BPF verifier | cargo ktstr verifier | Verifier |
| Analyze results, gate regressions | cargo ktstr stats / perf-delta | Runs |
| Narrow CI to affected schedulers | cargo ktstr affected | CI |
| Reproduce a test on bare metal | cargo ktstr export | this page |
| Debug interactively in a VM | cargo ktstr shell | this page |
Common flags
These flags mean the same thing on every subcommand that takes them.
--kernel ID — one grammar everywhere:
cargo ktstr test --kernel ../linux # local source tree (builds + caches)
cargo ktstr test --kernel 6.14.2 # version (auto-downloads on miss)
cargo ktstr test --kernel 6.14 # major.minor prefix → latest patch release
cargo ktstr test --kernel 6.14.2-tarball-x86_64-kc... # cache key (from `kernel list`)
cargo ktstr test --kernel 6.12..6.14 # range: every stable+longterm release inside
cargo ktstr test --kernel git+https://example.com/r.git#tag=v6.14 # git tag (#branch= / #sha= too)
cargo ktstr test --kernel fedora # prebuilt distro kernel (latest; fedora-44/f44 pin)
cargo ktstr test --kernel ubuntu # latest LTS HWE kernel (ubuntu-24.04 pins the LTS)
cargo ktstr test --kernel amazonlinux # AL2023's newest kernel stream (al2023 works too)
cargo ktstr test --kernel steamos # Valve's linux-neptune (x86_64 only; steamos-3.8 pins)
cargo ktstr test --kernel gke # latest GKE-promoted COS kernel (x86_64)
cargo ktstr test --kernel gke-129 # newest promoted revision within COS milestone 129
cargo ktstr test --kernel ./kernel-core.rpm # local package (.rpm / .deb / .pkg.tar.zst)
cargo ktstr test --kernel 6.14.2 --kernel 7.0 # repeatable → multi-kernel matrix
Ranges expand against kernel.org’s releases.json; both endpoints
are series-inclusive (6.11..6.14 covers every 6.14.N; spell
6.14.2 for an exact bound), and EOL series silently drop out
unless you pass --include-eol. Git sources are fetched at the ref,
built, and cached; a moved branch tip rebuilds.
Distro specs skip the build entirely: the kernel is resolved from
the distro’s official repo metadata (so a bare name always means the
latest published kernel — re-resolved every run), downloaded with
sha256 verification alongside its debuginfo, and cached like any
built kernel. Prebuilt kernels ship virtio as modules; ktstr
extracts the needed drivers from the package and loads them in the
guest before any device is touched, so shell, test, coverage,
and verifier all work unchanged. The package’s config is checked
on acquire: anything the kernel can’t boot without fails fast, and
missing ktstr-feature options (e.g. CONFIG_SCHED_CLASS_EXT on an
older kernel) warn up front instead of failing mid-run. Local
.rpm/.deb/.pkg.tar.zst files take the same path — note Fedora
and Ubuntu split the kernel image and its modules across packages,
so for those use the distro spec (which fetches the full set) rather
than a single file. GKE is the deliberate exception to the package
shape: ktstr downloads Google’s exact GKE-promoted bootable vmlinux,
matching prepared headers and source from cos-tools, then builds only
the missing virtio/Btrfs loadable modules against those exact artifacts.
The Google kernel is not rebuilt or modified; all downloads are
generation-pinned and MD5-verified before the normal atomic cache
install.
Bare gke is deliberately rolling: every resolution scans Google’s
current GKE release notes and chooses the numerically newest complete
cos-MILESTONE-BUILD-PATCH-REVISION image named there. This means the
newest GKE-promoted COS kernel, not specifically the Kubernetes Stable
release channel. gke-129 constrains the search to COS milestone 129
but still follows that milestone’s newest promoted build, patch, and
revision. For an immutable rerun, resolve once and use the resulting
generation-specific cache key shown by cargo ktstr kernel list.
Whichever form is used, ktstr refuses an image that lacks its required
sched_ext, BPF, debug, virtio, or module capabilities rather than
silently falling back to a reduced-capability kernel.
When --kernel resolves to two or more kernels, the kernel becomes
another gauntlet dimension: each (test × preset × kernel) tuple is a
distinct nextest case with a kernel-label suffix (see
test name shapes), and
result sidecars partition per kernel under
target/ktstr/{kernel}-{project_commit}/. Kernel resolution
finishes for every requested kernel before any test runs — a
missing kernel aborts up front rather than mid-matrix. host_only
tests run once regardless of kernel count.
--no-perf-mode — disable all
performance mode features
(flock, pinning, RT scheduling, hugepages, NUMA mbind, KVM exit
suppression). Also via KTSTR_NO_PERF_MODE.
--no-skip-mode — convert resource-contention and
host-topology-insufficient skips into hard failures (exit 1 instead
of 0). The default skips so a contended runner does not fail tests
that simply could not start.
The three profiles — independent of each other:
--release— the harness build profile. Release mode applies stricter assertion thresholds (gap_threshold_ms2000 vs debug’s 3000,spread_threshold_pct15% vs 35%), so tests that barely pass in debug may fail under--release.--profile NAME— the cargo build profile for the scheduler-under-test (a discovered scheduler package). Defaults torelease; pass--profile devfor a fast unoptimized scheduler build.--nextest-profile NAME— the nextest test profile from.config/nextest.toml(retries, timeouts, output settings).
For every nextest-backed route (test, coverage, verifier, replay
--exec, and raw llvm-cov nextest), cargo-ktstr supplies a
low-priority nextest tool config. It admits VM-resource waiters immediately
so ktstr’s cross-process topology queue—not a second CPU-sized nextest
queue—decides which fitting VM starts next, while ordinary host tests remain
bounded to the host CPU count. For orchestrated nextest runs, cargo-ktstr
normalizes every valid -j / --test-threads spelling to its effectively
unbounded admission budget; otherwise a caller-supplied CPU-sized value would
reintroduce a second scheduler ahead of ktstr. Repository policy unrelated to
the run-slot budget still applies. Raw llvm-cov modes such as report,
clean, and show-env are unchanged.
--relevant / --base / --base-ref / --default-branch —
narrow the run to only the tests whose scheduler your working-tree
change touches, against a baseline commit (merge-base with main by
default). A broad or unattributable change fails safe and runs
everything; a docs-only change runs nothing. The CI-matrix
counterpart is affected; both are documented in
CI.
test
Build the kernel (if needed) and run tests via cargo nextest run.
Also available as cargo ktstr nextest (a clap alias). Arguments
after -- are passed through to nextest:
cargo ktstr test --kernel ../linux # everything
cargo ktstr test --kernel 7.0 -- -E 'test(my_test)' # nextest filter
cargo ktstr test --kernel 7.0 -- --retries 2 # nextest retries
cargo ktstr test --kernel 7.0 -- --features integration # cargo features
cargo ktstr test --relevant # only tests my edits affect
Before a supported command runs nextest or builds/probes downstream ktstr
test registries, cargo-ktstr uses a Cargo metadata preflight to find direct
optional ktstr dependencies and their ktstr-only feature chains. It
automatically adds package-qualified selectors such as
--features scx_lavd/ktstr-tests. Ordinary commands restrict those
selectors to Cargo’s selected/default workspace members. Existing
--features are preserved; an explicit Cargo-side --all-features
remains authoritative.
This applies to test/nextest, coverage, llvm-cov nextest,
replay --exec, verifier, and the shared test-binary build used by
export, shell --test, and cargo ktstr affected. Commands that
neither run nextest nor build/probe workspace test binaries do no feature
inference.
Cargo metadata cannot identify arbitrary source-level cfg expressions.
Only ktstr-specific feature chains are inferred; composite gates and
transitive optional-helper arrangements remain explicit. Target-specific
optional ktstr dependencies are inferred only when they match the effective
Cargo target: cargo-ktstr asks rustc for that target’s complete cfg set and
evaluates the manifest platform expression before enabling the gate.
A real single-test run against a local kernel tree looks like this. The source tree was already built, so ktstr resolved it to a cache key and reused the image:
$ cargo ktstr test --kernel ../linux --no-perf-mode -- --features integration -E 'test(=ktstr/scx_empty_run_exits_under_watchdog)'
cargo ktstr: resolved kernel "path_linux_b5562c"
cargo ktstr: BTF type anchor at ~/ktstr/target/ktstr_btf_anchor.h
...
Finished `test` profile [unoptimized + debuginfo] target(s) in 27.24s
────────────
Nextest run ID d79da4e4-df49-4b40-a415-c1dff2739ce6 with nextest profile: default
Starting 1 test across 120 binaries (13597 tests skipped)
PASS [ 8.601s] (1/1) ktstr::scx_cleanup_test ktstr/scx_empty_run_exits_under_watchdog
────────────
Summary [ 8.642s] 1 test run: 1 passed, 13597 skipped
cargo ktstr: test outputs
~/ktstr/target/ktstr/7.1.0-486fe68-dirty
(1 stats sidecar(s), 0 wprof trace(s) written this run) — run `cargo ktstr stats last-run` for the gauntlet analysis
The gauntlet analysis (outlier scan, verifier stats, callback
profile, KVM stats) is no longer dumped after every run; the footer
points at stats last-run, which prints it on demand.
The VM part of this run took about nine seconds; the longer compile
line was Rust rebuilding the test harness. For a --kernel <path>
source tree, a cache hit is announced on stderr as
cargo ktstr: cache hit for {path} ({cache_key}, built {age} ago) and
skips the kernel build entirely.
Per-test exit codes
Each #[ktstr_test] process exits with one of three codes, so CI
gates and dashboards can triage runs:
| Code | Verdict | Meaning |
|---|---|---|
0 | Pass / Skip | Assertions passed, or the test never ran (host too small, perf mode unavailable). Skips degrade to pass unless --no-skip-mode. |
1 | Fail | An assertion failed; a retryable transient host-resource failure; an operator --cpu-cap the host cannot satisfy; a skip under --no-skip-mode; or expect_err = true and the test passed. |
2 | Inconclusive | A zero-denominator ratio gate could not evaluate — the workload produced no signal to ratio against. |
Exit code 2 is the silent-pass guard: a Pass at a ≤ threshold
gate run against a 0 / 0 ratio that synthesized to 0.0 would
have shipped a false-green CI run. The harness records Inconclusive
instead (see Checking) and
projects it to a distinct exit code so external tooling can route
the run separately from real regressions. The values are exported as
EXIT_PASS / EXIT_FAIL / EXIT_INCONCLUSIVE in the prelude.
replay
Re-run only the tests that failed in the last session, from the
result sidecars under target/ktstr/:
cargo ktstr replay # print the nextest filter (dry-run)
cargo ktstr replay --exec # actually run it
cargo ktstr replay -E crash # narrow by test-name substring
cargo ktstr replay --dir PATH # source sidecars from an archived tree
Dry-run is the default: the filter prints to stdout so you can
inspect it (or paste it into CI) before committing to the re-run.
Because sidecars record the registered bare test name, replay selects
every currently listed ordinary, host/, ktstr/, and gauntlet/ variant
of that name; absent variants simply match nothing.
--profile / --nextest-profile apply with --exec. Distinct from
auto-repro, which fires inside the failing test
process; replay is post-hoc, across a whole session.
coverage
Run tests with coverage via cargo llvm-cov nextest. Same kernel
resolution, multi-kernel semantics, and common flags as test;
arguments after -- pass through to cargo llvm-cov nextest:
cargo ktstr coverage --kernel ../linux
cargo ktstr coverage -- --workspace --lcov --output-path lcov.info
Requires cargo-llvm-cov and the llvm-tools-preview rustup
component. Guest-side coverage is flushed over shared memory at VM
exit and merged into the report automatically; multi-kernel runs
merge every variant’s profraw into a single report.
Managed cargo ktstr coverage runs merge producer/build-script shards into
one cached *.profdata; raw producer shards never enter the stable artifact
tree or content cache. After the tests, ktstr merges that seed with the live
runtime shards and removes the private raws only after cargo llvm-cov report
succeeds. A failed merge/report or an interrupted report retains the available
raws and merged profile below the ktstr cache’s
coverage-profraw-recovery-v1/ directory and prints the exact recovery path.
--no-report keeps cargo-llvm-cov’s ordinary user-owned artifact lifecycle.
Host-side files from plain
cargo ktstr test also remain next to the cargo-ktstr binary (an
LLVM_PROFILE_FILE injection keeps default.profraw out of your kernel
source tree; export LLVM_PROFILE_FILE yourself to opt out). To remove
retained shards manually:
cargo ktstr llvm-cov clean --profraw-only # only *.profraw under target/llvm-cov-target/
rm -f target/debug/llvm-cov-target/default-*.profraw
rm -f target/release/llvm-cov-target/default-*.profraw
Avoid bare cargo ktstr llvm-cov clean (wipes reports too) and
--workspace (also runs cargo clean).
llvm-cov
Raw passthrough to cargo llvm-cov for subcommands that don’t fit
the coverage flow (report, clean, show-env), with the same
kernel-resolution plumbing:
cargo ktstr llvm-cov report --lcov --output-path lcov.info
cargo ktstr llvm-cov nextest --workspace
Always pass a subcommand: a bare cargo ktstr llvm-cov falls
through to cargo test, which skips gauntlet variants and verifier
cells entirely (they exist only under the nextest harness). The explicit
nextest form gets targeted optional-ktstr feature inference; every
other raw form (test, run, report, clean, show-env, or bare)
preserves the supplied feature selection.
kernel
Manage cached kernel images: list, build, clean. The
standalone ktstr kernel subcommands are identical.
How kernels are discovered
There are two flows, and they intentionally differ:
Explicit (--kernel ...) — the full pipeline. For a source-tree
path: validate it is a kernel tree, look up the cache (clean git
trees only — a cache hit short-circuits straight to tests),
auto-configure (make defconfig if needed, append ktstr’s kconfig
fragment, make olddefconfig), build with
make -j$(nproc) KCFLAGS=-Wno-error, validate that the critical
config options survived (CONFIG_SCHED_CLASS_EXT,
CONFIG_DEBUG_INFO_BTF, CONFIG_BPF_SYSCALL, tracing options —
the kernel build system silently drops options with unmet
dependencies, and each failure gets a remediation hint), generate
compile_commands.json, store the result in the cache, and run
nextest with KTSTR_KERNEL (and KTSTR_KERNEL_LIST for
multi-kernel) exported. Dirty or non-git trees build in place, get a
_dirty kernel label, and never cache. Version, range, and git
identifiers download/fetch + build + cache on miss.
Implicit (no --kernel) — discovery only, no builds. The test
framework checks KTSTR_TEST_KERNEL, then KTSTR_KERNEL, then the
newest valid cache entry, then local build trees (./linux,
../linux, /lib/modules/$(uname -r)/build), then host images
(/boot/vmlinuz*). Whatever pre-built image it finds is used as-is
— nothing is compiled and nothing new lands in the cache.
Tip
If you iterate on a local kernel tree, pass
--kernel ../linux(or runcargo ktstr kernel build --kernel ../linuxonce) so the build is cached and reused. Implicit discovery will find the tree but never populate the cache for it.
kernel list
List cached kernels, newest first:
$ cargo ktstr kernel list
cache: ~/.cache/ktstr/kernels
KEY VERSION SOURCE ARCH BUILT
7.0.14-tarball-x86_64-kcabd40422 7.0.14 tarball x86_64 2026-07-04T23:06:12Z
local-b4dc42d-x86_64-kcabd40422 7.1.0 local x86_64 2026-07-04T23:00:04Z
local-5123e5a-x86_64-cfg1982ad42-kcabd40422 7.1.0 local x86_64 2026-07-04T22:20:13Z
...
local-c5d2724-x86_64-kcabd40422 7.0.0 local x86_64 2026-07-04T22:05:35Z
...
7.0.13-tarball-x86_64-kc5f2631f0 7.0.13 tarball x86_64 2026-06-22T21:54:56Z (stale kconfig)
7.1.1-tarball-x86_64-kc5f2631f0 7.1.1 tarball x86_64 2026-06-22T21:46:19Z (stale kconfig)
warning: entries marked (stale kconfig) were built against a different ktstr.kconfig. Rebuild with: kernel build --force --kernel <entry version> (add --extra-kconfig PATH if the entry also carries the (extra kconfig) tag).
(stale kconfig) marks entries built against a different ktstr
kconfig fragment (they are rebuilt automatically when requested);
(EOL) marks entries whose series has left kernel.org’s active
releases list. --json emits the same data for scripting. With
--kernel START..END, list switches to preview mode: it prints
the versions the range expands to without downloading or building
anything — the cheap answer to “what does 6.12..6.16 actually
cover?”.
kernel build
Download (or use a source tree / git ref), build, and cache one kernel or a range:
cargo ktstr kernel build # latest stable from kernel.org
cargo ktstr kernel build --kernel 6.14.2 # specific version
cargo ktstr kernel build --kernel 6.12 # latest 6.12.x patch release
cargo ktstr kernel build --kernel 6.11..6.14 # every release in the range
cargo ktstr kernel build --kernel ../linux # local source tree
cargo ktstr kernel build --kernel gke # official GKE COS kernel + exact-source modules
cargo ktstr kernel build --kernel gke-129 # newest promoted revision in COS milestone 129
cargo ktstr kernel build --force --kernel 6.14.2
With no --kernel, it builds the latest stable series that has had
at least 8 maintenance releases — keeping CI off brand-new majors
whose early builds are more likely to break. Already-cached entries
are skipped unless --force.
| Flag | Description |
|---|---|
--force | Rebuild even if cached. |
--clean | make mrproper first (source-tree builds only). |
--cpu-cap N | Reserve exactly N host CPUs for the build (build parallelism and a cgroup sandbox follow). See Resource Budget. |
--extra-kconfig PATH | Extra kconfig fragment merged over ktstr’s (user values win); lands in its own cache slot. |
--skip-sha256 | Skip tarball checksum verification (emits a warning). |
--include-eol | With a range, also build EOL series from the linux-stable mirror. |
A bare relative name is read as a cache key — prefix a relative
source directory with ./.
kernel clean
cargo ktstr kernel clean --keep 3 # keep the 3 most recent valid entries
cargo ktstr kernel clean --corrupt-only --force # drop only broken entries
Corrupt entries (missing metadata or image) never consume a --keep
slot. --force skips the confirmation prompt (required
non-interactively).
verifier
Sweep every declare_scheduler!-registered scheduler through the
real kernel’s BPF verifier across topologies, checking verify +
attach + dispatch per cell. Takes the same --kernel grammar as
test — repeatable, ranges (6.12..6.14, with --include-eol for
end-of-life series), paths, cache keys, git refs:
cargo ktstr verifier --kernel 7.0
cargo ktstr verifier --scheduler scx-ktstr # one scheduler
cargo ktstr verifier --raw # no cycle collapse
Unlike ordinary test commands, a bare/unscoped verifier recursively
widens its Cargo package selection to every compatible workspace package.
Explicit package selectors remain scoped to the requested packages. The
verifier reuses the targeted feature inference described under
test, while excluding packages linked to an older ktstr that
this verifier dispatcher cannot enumerate.
See BPF Verifier Sweep for the cell model, real output, and the kernels-filter contract.
shell
Boot any kernel ktstr can resolve and land in a busybox shell inside the VM — the fastest way to poke at a kernel you just built, check what a guest actually exposes, or reproduce a failing test’s environment by hand:
cargo ktstr shell --test my_failing_test # boot the exact VM a test gets
cargo ktstr shell --kernel ../linux # local tree
cargo ktstr shell --kernel ./arch/x86/boot/bzImage
--test NAME derives topology, memory, and include files from a
registered #[ktstr_test] (mutually exclusive with --topology /
--memory-mib; -i is additive) — so “why does this only fail on
9cpu-3llc-nosmt” becomes an interactive session in that exact machine.
For scripted checks, --exec runs a command and exits with its
status:
$ cargo ktstr shell --kernel ../linux --no-perf-mode \
--exec 'uname -r; cat /sys/kernel/sched_ext/state; nproc'
7.1.0-00001-g8cd2b47fb188
disabled
1
The standalone ktstr shell is the
same boot flow minus the cargo integration (--test, raw image
paths).
export
Export a registered test as a self-extracting .run file that
reproduces the scenario on bare metal, no VM: the ktstr binary, the
scheduler binary, and every declared include file, embedded in a
bash preamble that validates root, sched_ext support, cgroup2, the
no-other-scheduler-attached invariant, and topology compatibility
before launching.
cargo ktstr export my_test -o /tmp/my_test.run
wrote /tmp/sched_basic_proportional.run (90074903 bytes archive, 0 include files)
The generated script opens with the frozen test specification and the preflight checks:
#!/bin/bash
# Generated by `cargo ktstr export`. Do not edit; regenerate to update.
set -euo pipefail
# --- frozen test specification ---
KTSTR_TEST_NAME=sched_basic_proportional
KTSTR_SCHED_NAME=ktstr_sched
KTSTR_GIT_HASH=73730e0
NEED_LLCS=1
NEED_CORES_PER_LLC=2
NEED_THREADS_PER_CORE=1
NEED_NUMA_NODES=1
TEST_DURATION_SECS=12
TEST_WATCHDOG_SECS=15
Scheduler choice, scheduler args, and topology are frozen at export
time; --duration, --watchdog-timeout, and --quiet can be
overridden at .run invocation. --package NAME scopes the
workspace search (and disambiguates duplicate test names —
otherwise the first matching binary in path order wins);
--release builds the embedded binaries with the release profile to
match how you will run them.
Not exportable (rejected with actionable errors): host_only tests,
tests using bpf_map_write (they need the host-side probe surface),
and KernelBuiltin schedulers.
completions
cargo ktstr completions bash >> ~/.local/share/bash-completion/completions/cargo
cargo ktstr completions zsh > ~/.zfunc/_cargo-ktstr
cargo ktstr completions fish > ~/.config/fish/completions/cargo-ktstr.fish
SHELL is one of bash, zsh, fish, elvish, powershell;
--binary overrides the completion target (default cargo).
stats
Sidecar analysis for past runs: stats last-run (the gauntlet
analysis — outlier scan, verifier stats, callback profile, KVM stats
— of the newest run), stats list (run table), stats list-metrics
(the regression metric registry), stats list-values (distinct
filter values in the pool), stats show-host --run ID (archived host
context), and stats explain-sidecar --run ID (why optional fields
are absent). Bare cargo ktstr stats prints this subcommand list.
cargo ktstr stats last-run # newest run's analysis
cargo ktstr stats last-run --kernel 7.1.3 # newest run for that kernel
cargo ktstr stats last-run --dir PATH # a specific run dir / archive
last-run resolves its source dir in precedence order: --dir, then
KTSTR_SIDECAR_DIR, then --kernel (newest run dir whose leaf begins
{LABEL}-), then the most-recently-modified run dir.
cargo ktstr show-host prints the same host context live, and
cargo ktstr show-thresholds TEST prints the merged assertion
thresholds a test will run with. See
Runs and Regression Gates.
perf-delta
Compare performance_mode test metrics between HEAD and a baseline
commit, exiting non-zero when enough metrics regress to trip the
gate:
cargo ktstr perf-delta --noise-adjust 5 --kernel 6.14
Baseline resolution, the --noise-adjust statistics, and the full
flag set are documented in
Runs and Regression Gates; the worked
walkthrough is A/B Compare Branches.
affected
Emit the scheduler packages a base..HEAD diff affects, as a JSON
array for a GitHub Actions dynamic matrix
(["scx_lavd","scx_rusty"]). Attribution is fail-safe: any
uncertainty widens to the full set, and only a strictly docs-only
change emits []. --relevant is the same engine applied locally
to narrow test / coverage / perf-delta runs. Both are
documented with the complete CI workflow in CI.
locks
Enumerate every ktstr flock held on this host — read-only, never
acquires anything. When a build or test is blocked behind a peer’s
reservation, cargo ktstr locks names the peer without disturbing it:
$ ktstr locks
LLC locks:
LLC NODE LOCKFILE HOLDERS
0 0 /tmp/ktstr-llc-0.lock <none recorded>
1 0 /tmp/ktstr-llc-1.lock <none recorded>
...
Run-dir locks:
RUN KEY LOCKFILE HOLDERS
7.0.14-73730e0-dirty target/ktstr/.locks/7.0.14-73730e0-dirty.lock <none recorded>
An idle host shows <none recorded>; while a lock is held, the
HOLDERS column names the holder’s PID and cmdline
(cross-referenced against /proc/locks). Four lock-file roots are
scanned:
{KTSTR_LOCK_DIR}/ktstr-llc-*.lock(default/tmp) — per-LLC reservations held by perf-mode test runs and--cpu-cap-bounded builds.{KTSTR_LOCK_DIR}/ktstr-cpu-*.lock— per-CPU reservations from the same flow.{cache_root}/.locks/*.lock— kernel-cache entry locks held duringkernel buildwrites, plus per-source-tree locks held while building from a path.{runs_root}/.locks/{kernel}-{project_commit}.lock— sidecar write locks serializing concurrent runs targeting the same run directory.
| Flag | Default | Description |
|---|---|---|
--json | off | JSON snapshot (pretty in one-shot mode; ndjson under --watch). |
--watch DURATION | unset | Redraw at the interval until SIGINT (100ms, 1s, 5m). |
Available identically as the standalone ktstr locks. The
reservation model behind these locks is documented in
Resource Budget.
The standalone ktstr binary
cargo install ktstr also installs a bare ktstr — the same engine
without the cargo front-end. It is the guest-init binary (it runs as
PID 1 inside every test VM) that doubles as a host CLI for the tools
that don’t need a cargo project: ktstr topo, ktstr locks,
ktstr ctprof, ktstr kernel, and ktstr shell. Each mirrors its
cargo ktstr counterpart.
Reach for it when you are debugging outside a cargo project — poking
at a kernel you just built, or inspecting a host with no ktstr
workspace checked out. The common case is ktstr shell, an
interactive KVM VM on any kernel ktstr can resolve:
ktstr shell --kernel ../linux
ktstr shell --kernel 6.14.2 --topology 1,2,4,1
ktstr shell -i ./my-binary -i strace --exec 'cat /proc/schedstat'
Files passed with -i land at /include-files/<name> inside the
guest (directories walked recursively, dynamically-linked ELF
binaries get their shared-library closure resolved). It takes the
same --kernel grammar as cargo ktstr test, plus
--topology N,L,C,T, --memory-mib, --disk, --dmesg, and
--exec.
cargo ktstr is the primary interface: it resolves kernels from your
workspace, accepts raw kernel-image paths, and adds --test NAME to
reuse a registered test’s exact VM shape. Drop to ktstr only when
you are not inside a project.