Long-Horizon-Terminal-Bench (LHTB) was submitted to arXiv on July 9, 2026 (paper 2607.08964, revised July 13) by a 13-author team led by Zongxia Li. It is a benchmark of 46 long-horizon terminal tasks spanning nine categories, including experiment reproduction, software engineering, multimodal analysis, interactive games, and scientific computing. Unlike one-shot coding benchmarks, LHTB tasks typically require hundreds of episodes and minutes to hours of execution, stressing sustained planning, long-context management, and iterative debugging.
The benchmark’s distinguishing feature is dense reward-based grading. Rather than a binary pass or fail on the final state, tasks are decomposed into subtasks that award intermediate rewards and partial credit, giving a finer-grained view of how far an agent progressed. The authors report that agents consume on average 9.9 million tokens per task, run roughly 231 episodes, and take 85.3 minutes of execution time per run, making LHTB substantially more demanding than prior terminal benchmarks.
Results expose a wide capability gap. The strongest tested model reaches only 15.2 percent pass@1 at a 0.95 reward threshold and 10.9 percent at a perfect 1.0 threshold, while the mean across models is 4.3 percent and 1.7 percent respectively. The benchmark, dataset, and code were released publicly on GitHub and Hugging Face.
For technical leaders betting on autonomous coding and operations agents, LHTB is a reality check: current frontier agents still stall badly on realistic, multi-hour terminal work, and partial-credit scoring makes those failures measurable rather than hidden behind a single pass rate.