Architecture Overview
A scheduler bug rarely returns an error code — it wedges a CPU, strands a runqueue, or panics the kernel. ktstr’s architecture follows from that: every test boots its own KVM microVM, so a crash takes down a disposable guest instead of your machine; the CPU topology is whatever the test declared; and the kernel is the exact build you targeted.
ktstr has three execution domains:
The test binary on the host manages VM lifecycle, monitors guest memory, and evaluates results.
The same binary runs inside the VM as PID 1: mounts filesystems, starts the scheduler, creates cgroups, forks workers, and writes results back.
A host thread reads guest VM memory directly, observing scheduler state without instrumenting the guest.
Execution flow
The host builds an initramfs that contains the same test binary as /init, plus the scheduler binary, payloads, and files the test declared.
KVM starts a disposable VM using the requested kernel, topology, memory, transports, and performance-mode settings.
Inside the VM, the test binary takes the guest path: mount filesystems, read the host-provided test spec, and prepare cgroup state.
A host thread begins reading guest memory through KVM mappings, resolving BTF offsets and watching scheduler state without injecting code into the guest.
The guest starts the scheduler, creates cgroups and cpusets, forks workers, applies ops, and records worker reports.
The monitor records stuck observations, snapshots, event counters, dumps, and scheduler-exit context while the workload runs.
The guest evaluates worker checks and writes the structured result to virtio-console port 1.
The host combines guest results with monitor findings, writes artifacts, and returns the nextest case outcome.
Results travel on virtio-console port 1; panics, crashes, and other non-blockable diagnostics fall back to the COM2 serial port (see VMM — guest–host transports).
Key design decisions
Same binary, two roles. The test binary serves as both host
controller and guest test runner. The initramfs embeds the binary as
/init; when the binary finds itself running as PID 1, it executes
the guest lifecycle (mounts, scheduler start, test dispatch, reboot)
instead of the host one. One cargo build produces everything needed
for both sides — there is no separate guest agent to version or ship.
Forked workers (default), threads optional. The default Fork
clone mode spawns each worker as its own process so cgroup placement
via cgroup.procs is tgid-granular. The Thread clone mode shares
the harness’s tgid and routes placement through cgroup.threads
instead — useful when workers need a shared address space or when
measuring thread-only scheduler paths. See
Workers and Workloads.
Host-side monitoring. The monitor reads guest memory via KVM, avoiding BPF instrumentation of the scheduler under test. This eliminates observer effects on scheduling decisions.
Where to go next
- VMM — how VMs boot, topology modeling, guest–host transports.
- Monitor — what is observed from the host and how violations become verdicts.
- Workers and Workloads — worker lifecycle and the telemetry each worker reports.
- CgroupManager and CgroupGroup — cgroup plumbing and RAII cleanup inside the guest.