Submitted to arXiv on August 24, 2026 (paper 2608.23564) by Deyao Hong, Yizhe Chi, Wenyi Li, Xiaoqiu Wang, Mingju Gao, Kaisen Yang, Bingxiang He, Youjie Zheng, Calvin Xiao and Qinhuai Na. The question is one every engineering organisation with a decade of accumulated technical debt eventually asks: coding agents are good at fixing bugs, so can they carry out the expensive, largely manual work of migrating a whole repository off an old stack?
Existing benchmarks cannot answer it, because they check behavioural correctness only. The authors name the resulting exploit Blindness - an agent copies the original implementation forward so the tests pass, and scores full marks without performing the migration at all. SWE Refactor Bench closes that hole with 20 whole-repository migrations across four kinds of technical debt and a three-stage protocol: a Migration Audit that verifies the migration actually happened, a fixed suite of Behavioural Tests, and Agentic Verification in which six independent coding agents generate targeted tests hunting for hidden behavioural differences.
Across 520 runs spanning 8 frontier models and 26 model-effort configurations, only 28 runs, or 5.4 percent, passed all three stages. Thirteen of the 20 tasks received no accepted solution at all, and the best model, claude-opus-5, scored 47.0 out of 100. The two failure modes separate cleanly: a few runs preserved behaviour by skipping the migration and were caught at the Migration Audit, while most attempted the migration and broke behaviour, stopping at the Behavioural Tests. Migration completeness and behavioural correctness are, on this evidence, distinct abilities.
The last-mile result is the one to plan around. Among the 340 runs that passed the Migration Audit, 58 percent reached 99 percent of the fixed checks but only 26 percent reached 100 percent - agents get almost all the way and then stall, which is precisely the shape of work that costs the most to finish by hand. Capability also varies sharply by category: 31.4 on build toolchain rewrites against just 5.6 on language rewrites. Budgeting an agent-led framework upgrade is defensible today; budgeting an agent-led language port is not.