ScarfBench

Background

Framework migration requires coordinated changes across an application rather than an isolated patch. ScarfBench measures whether agents can carry those changes through while preserving externally visible behavior.

What it evaluates

• Understanding an existing enterprise application
• Translating framework-specific components and configuration
• Coordinating changes across files and layers
• Preserving buildability and behavior

Evaluation methodology

Agent submissions are evaluated as complete migrated applications using executable build and correctness checks.

Representative task

Migrate a working Java application from its current framework to a specified target framework while retaining its required behavior and passing the validation suite.

Results

The paper describes the task set, evaluation harness, and baseline performance.