When an AI agent says a task is done, that is a claim, not proof. Microsoft's ThinkingBox benchmark, published with Hugging Face in October 2026, grades agents on the records they leave behind in backend systems across 507 business workflows, each run 20 times.
The finding that matters for operators: in a 121,680-trial ablation across 12 models, 67.24% of failed attempts still ended cleanly, called a state-changing tool, and reported no final tool error. Executable checks then found wrong field values in 77.61% of those failures.
Consistency is a separate question from capability. In the benchmark, Claude Opus 5 passed 47.53% of tasks on all 20 attempts, while Kimi-K3, which solved 93.89% of tasks at least once, passed only 13.41% on every attempt.
Practical test plan: pick 10–20 real tasks, define the required end state in your systems (ticket status, refund amount, CRM field), run each task several times, and score only the end state. The authors also recommend checking terminal state before committing, classifying tool errors so retries target recoverable ones, narrowing the tool surface, and requiring human approval for changes you can't cheaply reverse.