This article covers 2026-07-06 through 2026-07-12 in Asia/Taipei time. The research signal was not another average score. It was an explanation of why agents that look capable on short tasks collapse over long ones: tool errors, hidden state, accumulated context, coordination, and measurement flaws compound.

Six failure clusters across benchmarks

Beyond the Leaderboard, posted July 7, synthesizes 27 benchmark, taxonomy, and audit papers into six clusters: tool/parameter errors; planning and constraints; long-horizon context degradation; multi-agent coordination; safety; and measurement validity. Its central claim is that errors compound nonlinearly with task length, so strong subtasks do not imply reliable completion.

This is a synthesis, not a new unified experiment. Its categories depend on the included literature and how heterogeneous definitions are aligned. Its practical value is an incident taxonomy richer than “the model was wrong.”

OSWorld 2.0 stretches computer use to hundreds of actions

OSWorld 2.0 introduces 108 long-horizon computer workflows with a median human completion time around 1.6 hours and potentially hundreds of agent tool calls. The authors report that the best system completes only about 20.6% under the primary 500-step binary metric. Agents miss information arriving mid-task, guess instead of asking, skip verification, and lose hidden state.

The benchmark is more realistic than single-click tasks but remains controlled. Real organizations add permissions, latency, notifications, collaboration, and irreversible actions. Its best use is testing checkpoints, state recovery, asking policies, and verification—not claiming job-level replacement.

PlanBench-XL makes recovery part of planning

PlanBench-XL contains 327 retail tasks over 1,665 tools and injects missing, failing, or distracting functions. The paper reports a leading model at roughly 51.9% without blocking and about 11.36% under the harshest blocking. Failures are worst when no explicit error appears or recovery needs a longer alternate path.

The enduring lesson is to evaluate tool retrieval, execution, and recovery together. Production traces should capture candidate tools, parameters, responses, error classes, alternate plans, and stop reasons.

Synthesis: reliability lives at the trajectory layer

Together, the papers show why a stronger base model is insufficient. Long tasks need checkpoints, a constraint ledger, state refresh, failure classification, recoverable tools, and independent verification. Multi-agent scaffolding also introduces handoff and shared-state failure surfaces.

Watchlist

• Independent reruns of OSWorld 2.0 tasks, artifacts, and safety reports.
• PlanBench-XL extensions into finance, healthcare, and developer tools.
• Turning failure taxonomies into shared trace schemas and incident dashboards.
• Full-completion reporting instead of partial scores alone.
• Measured cost/success tradeoffs for verifiers, checkpoints, and escalation.