kBench & kGym

Background

Kernel crashes are difficult to evaluate at scale because reproducing them requires the right kernel configuration, environment, and workload. kBench supplies curated bugs; kGym supplies the experimentation infrastructure needed to boot, execute, and validate them.

What it evaluates

• Kernel crash reproduction
• Root-cause analysis
• Patch generation and iterative testing
• Use of execution evidence during debugging

Dataset and environment

Each kBench item pairs a real Linux kernel bug with a developer-provided fix and deterministic reproduction script. kGym provides isolated environments, execution traces, crash states, and parallel testing across kernel configurations.

Evaluation methodology

An agent is evaluated on whether it can reproduce the target failure and produce a patch that passes the task’s validation procedure.

Representative task

Reproduce a supplied kernel crash in kGym, analyze the resulting traces, propose a source patch, and validate the repair in a clean environment.

Results

The NeurIPS paper describes the dataset, platform, and reported evaluation results.