Real-world AI agents need evaluations that reflect the environments where they will actually operate. DAPLab builds benchmarks for complex, stateful tasks across software, data systems, and human-facing workflows—with verifiable outcomes and an emphasis on contamination-resistant evaluation.
A realistic, contamination-free benchmark for agentic reverse engineering. Programs are built from scratch by RE experts and hardened with in-house anti-analysis protections. Frontier agents fully solve only about a third of instances or fewer.
A benchmark for interactive U.S. federal individual tax filing, built on field-level operationalization of tax law. Agents must identify missing taxpayer information, query a simulated taxpayer, consult IRS guidance, and complete the required forms. Filing errors are traced back to the rules and sources involved.
Evaluates whether deep-research agents can discover and reason over relevant structured datasets in a million-scale data lake, then produce verifiable answers with explicit provenance.
Continuously draws from newly reported Linux kernel crashes to test whether agents can diagnose and resolve real, post-cutoff failures without relying on memorized fixes.
Measures whether branchable databases can support the creation, exploration, and pruning patterns of agentic workloads across configurable branch shapes, schemas, and operations.
Pairs a curated dataset of real Linux kernel bugs with deterministic reproduction scripts and a sandboxed experimentation platform for evaluating agent-driven crash diagnosis and patching.
A benchmark dataset for building and evaluating digital twins based on real people, designed to study how faithfully agents capture individual behavior and variation across domains.
A versatile user-simulation toolkit for evaluating interactive agents through realistic multi-turn behavior, reducing dependence on costly human evaluation.
Generates diverse, realistic user queries specifically designed to expose QA-agent failures, supporting more rigorous evaluation than static, average-case question sets.
Evaluates whether coding agents can migrate enterprise Java applications across frameworks while preserving behavior and handling changes that span a real application.
Diagnoses why coding agents can still fail after reaching the correct region of a codebase, exposing breakdowns in maintaining coherent reasoning and edits over time.
Tests whether agents can find, reproduce, and repair real concurrency bugs whose failures depend on particular interactions among threads, timing, and shared state. Each bug includes a specific oracle for validating both reproduction and repair.