Getting Started
Every #[ktstr_test] boots a real Linux kernel in a KVM microVM with
the CPU topology the test declares, runs your workload inside it, and
checks the scheduler’s behavior from the host. This page takes you
from nothing to a green run.
cargo-nextest runs the suite; ktstr installs both cargo-ktstr and the standalone host CLI.
Use a released version, local source tree, or cached build. The guest kernel needs sched_ext.
Start under EEVDF, then add scheduler = MY_SCHED when your scheduler binary is ready.
Zero to green
cargo install --locked cargo-nextest
cargo install --locked ktstr
cargo ktstr kernel build --kernel 7.0
$EDITOR tests/sched_test.rs
cargo ktstr test --kernel 7.0
Both installs are required: cargo ktstr test delegates to nextest,
and the ktstr package installs cargo-ktstr (the cargo plugin
behind every command in this guide) plus the standalone ktstr host
CLI. The kernel build is a real make -j$(nproc) kernel build —
plan for that once; later runs reuse the cache. On a cached kernel,
the run shown below took about nine seconds end to end.
The shortest useful test file looks like this:
use ktstr::prelude::*;
#[ktstr_test(llcs = 1, cores = 2, threads = 1)]
fn smoke(ctx: &Ctx) -> Result<AssertResult> {
scenarios::steady(ctx)
}
That smoke test runs under the kernel default scheduler. It is still a real VM run, so it proves KVM, kernel resolution, initramfs creation, guest boot, workers, and result transport before you add a custom scheduler.
Prerequisites
Linux only (x86_64, aarch64). ktstr boots KVM virtual machines; it does not build or run on other platforms.
You need:
- KVM access (
/dev/kvm) — see Troubleshooting if it’s missing or unreadable - Rust ≥ 1.94.1 (the crate’s MSRV)
- clang, pkg-config, make, gcc, and autotools (autoconf, autopoint, flex, bison, gawk) — BPF skeletons and the vendored libbpf/libelf/zlib build
- BTF (
/sys/kernel/btf/vmlinux) — present by default on most distros - Internet access on first build (downloads busybox source; kernel builds download tarballs from kernel.org)
The host kernel only needs KVM. The guest kernel — the one your tests boot — needs sched_ext, which landed in 6.12; the next section builds one.
Install distro packages once:
# Ubuntu/Debian
sudo apt install clang pkg-config make gcc autoconf autopoint flex bison gawk
# Fedora
sudo dnf install clang pkgconf make gcc autoconf gettext-devel flex bison gawk
Add the dependency
[dev-dependencies]
ktstr = "=0.26.0"
ktstr is pre-release: pin the exact patch version and keep the
installed cargo-ktstr on the same one — minor bumps may break the
test-facing API. To keep ktstr out of a scheduler crate’s normal
builds, gate it behind a feature instead — see
Test a New Scheduler.
Install the matching CLI when pinning a specific crate version:
cargo install --locked --bin cargo-ktstr ktstr@0.26.0
Build a kernel
cargo ktstr kernel build downloads a kernel tarball from
kernel.org, applies the embedded ktstr.kconfig fragment (sched_ext,
BPF, kprobes, minimal boot), builds it, and caches the result:
cargo ktstr kernel build # latest stable series with >= 8 point releases
cargo ktstr kernel build --kernel 7.0 # highest 7.0.x release
cargo ktstr kernel build --kernel 6.14.2 # exact version
cargo ktstr kernel build --kernel ../linux # local source tree
Choose the kernel spec that matches your workflow:
| Spec | Use it when |
|---|---|
--kernel 7.0 | You want the newest cached or buildable 7.0.x release. |
--kernel 6.14.2 | You need an exact released version. |
--kernel ../linux | You are testing a local kernel tree. |
no --kernel | You want ktstr’s discovery chain to choose. |
The bare kernel build form skips series with fewer than 8 maintenance
releases; brand-new majors tend to hit build issues on older toolchains.
Name a version explicitly to override. cargo ktstr kernel list shows
the cache and cargo ktstr kernel clean --keep 3 prunes it. You can
also skip this step entirely: cargo ktstr test --kernel 7.0 builds
and caches on first use.
Write a test
One mental model before the first example: your test function runs
inside the VM, as the guest’s init process. execute_defs and
friends create real cgroups and spawn real workers; ctx hands you
the guest topology (ctx.topo) and cgroup management
(ctx.cgroups).
Create a file in your crate’s tests/ directory (e.g.
tests/sched_test.rs). The simplest test runs a canned scenario:
use ktstr::prelude::*;
#[ktstr_test(llcs = 1, cores = 2, threads = 1)] // llcs = last-level caches
fn my_test(ctx: &Ctx) -> Result<AssertResult> {
// Canned scenario: two cgroups of CPU spinners, default duration.
scenarios::steady(ctx)
}
No scheduler attribute means the test runs under the kernel’s
default EEVDF scheduler (see Overview) — a useful
baseline before pointing at your own.
When the canned scenarios stop being enough, declare your own
cgroups, workloads, and cpusets with CgroupDef — the
Tutorial builds that up one step at a time, and
Writing Tests is the reference.
Point it at a sched_ext scheduler
Declare your scheduler once and reference it from any test:
use ktstr::prelude::*;
declare_scheduler!(MY_SCHED, {
name = "my_sched",
binary = "scx_mysched", // your scheduler's binary name
});
#[ktstr_test(scheduler = MY_SCHED, llcs = 2, cores = 2, threads = 1)]
fn my_sched_steady(ctx: &Ctx) -> Result<AssertResult> {
scenarios::steady(ctx)
}
The binary is resolved on the host — a KTSTR_SCHEDULER=/path
override, else built via cargo build -p <name> in the declaring
crate’s workspace — and packed into the VM’s initramfs. Full field
reference:
Scheduler Definitions;
walkthrough: Test a New Scheduler.
Run it
cargo ktstr test --kernel 7.0 # everything
cargo ktstr test --kernel 7.0 -- -E 'test(my_test)' # one test (nextest filter)
cargo ktstr test resolves the kernel — an explicit --kernel
version, path, or cache key, or, without the flag, a discovery chain
through environment variables, the kernel cache, and host kernels —
then wraps cargo nextest run. The full chain and flag grammar live
in cargo ktstr.
Here is a real run against a local kernel tree (transcript captured
from ktstr’s own suite — your run shows ktstr/my_test on the PASS
line instead):
cargo ktstr: resolved kernel "path_linux_b5562c" cargo ktstr: BTF type anchor at target/ktstr_btf_anchor.h ... ──────────── Nextest run ID d79da4e4-df49-4b40-a415-c1dff2739ce6 with nextest profile: default Starting 1 test across 120 binaries (13597 tests skipped) PASS [ 8.601s] (1/1) ktstr::scx_cleanup_test ktstr/scx_empty_run_exits_under_watchdog ──────────── Summary [ 8.642s] 1 test run: 1 passed, 13597 skipped cargo ktstr: test outputs ... (1 stats sidecar(s), 0 wprof trace(s) written this run)
Reading it:
- The first two lines are kernel resolution and BTF setup:
--kernel ../linuxused a local source tree and found the matching image in ktstr’s cache — no rebuild. - Test names have the shape
crate::binary ktstr/test_name; thektstr/prefix marks the base variant, and the same test also generatesgauntlet/topology variants, skipped by default (see Running Tests). The 8.6 s covers everything: VM boot, scenario, teardown, evaluation. - Every run writes a stats sidecar per test under
target/ktstr/{kernel}-{commit}/— the raw material for regression gates (Runs and Regression Gates).
What gets checked
A scheduler that dies fails the test by default: a crash, a load failure, or a stall (the kernel watchdog kicks a stalled scheduler out with an error) all exit non-zero and come back red, with the failure trail from Reading Failure Output.
Warning
Behavioral checks are a different story: nothing is asserted by default. A bare
#[ktstr_test]reports pass even if the scheduler gave some workers no measured work or spread CPU time wildly — as long as it stayed alive.
Every behavioral check is an opt-in attribute: not_stuck = true
enables the zero-work-units, fairness-spread, and stuck-gap worker
checks, max_spread_pct, min_iteration_rate, and friends set
explicit thresholds. Checking explains the model;
Customize Checking shows the override
flow.
When a check fails
A failing check prints the violated threshold with the observed
value, then per-cgroup statistics. This excerpt is from a real run
that set an impossible min_iteration_rate floor:
ktstr_test 'throughput_gate' [sched=scx-ktstr] [topo=1n1l2c1t] failed: worker 71 iteration rate 41903.3/s below floor 50000000.0/s worker 73 iteration rate 37834.5/s below floor 50000000.0/s --- stats --- 2 workers, 4 cpus, 2 migrations, worst_spread=0.0%, worst_gap=21ms cg0: workers=1 cpus=2 spread=0.0% gap=10ms migrations=1 iter=209600 cg1: workers=1 cpus=2 spread=0.0% gap=21ms migrations=1 iter=189252 ...
The header names the test, scheduler, and topology variant; each
detail line names the check, observed value, and threshold. The full
output continues with timeline, scheduler-log, and monitor sections,
plus failure-dump artifacts and a ready-to-paste cargo ktstr replay command — Reading Failure
Output walks the whole anatomy.
Next steps
- Tutorial: Zero to ktstr — build a complete test step by step, break it on purpose, and read the wreckage.
- Test a New Scheduler — you have
an
scx_*binary and want it under test in five minutes. - Writing Tests — the authoring reference: attributes, scenarios, snapshots, assertions.