Submitted to arXiv on August 14, 2026 (paper 2608.14036), “Demystifying Agent Skills: Why They Work - Until They Don’t” by Zhiyuan Jiang, Fangrui Huang, Hanwen Xing, Xander Wu, Yipeng Gao, Rui Cao, Mengdi Wang, Shilong Liu and Yijiang Li examines a practice that has spread across agent frameworks in 2026 faster than anyone has measured it. Skills - reusable written procedures an agent can retrieve and follow at inference time - are now a first-class construct in most major agent platforms. This paper asks the obvious unasked question: when do they actually help, and why do they fail.
The study is empirical rather than architectural. The authors run systematic experiments across multiple benchmarks and frameworks and analyze 8,135 trial records. Their central finding is that skills work mainly as procedural anchors that stabilize execution, a characterization they say accounts for 65.7 percent of observed cases - the skill keeps the agent on a coherent path rather than supplying knowledge it lacked. In matched comparisons they report a 6.06 point improvement over Workflow Memory. They also find that an agent does not need to invoke the ground-truth skill to succeed, which undercuts evaluations that score skill systems by retrieval correctness.
The failure analysis is the more operationally useful half. Retrieval, not authoring, is the bottleneck: reported precision falls from 29.6 percent to 3.3 percent as the skill pool grows. Skills fail, in the authors’ framing, when they carry brittle assumptions, land in an incompatible context, or are not adapted enough to the situation at hand. That is a recognisable description of what happens to any organisation’s internal runbooks over time, and it suggests skill libraries decay the same way documentation does.
For anyone running agents in production, the practical implication is to treat a skill library as a retrieval system with a maintenance burden, not as free capability that accumulates. A tenfold drop in retrieval precision as the library grows means the naive strategy - let every team add skills - actively degrades the agent past some size. Curation, scoping skills to contexts, and measuring retrieval precision as a first-class metric are the countermeasures the paper points toward. The results are from academic benchmark settings rather than production traffic, so the exact numbers should not be transplanted, but the shape of the failure is the part worth planning around.