The Agent Said It Was Done. The Database Disagreed.
Microsoft’s ThinkingBox benchmark evaluates AI agents by grading the terminal database state and side‑effects they leave behind, running each of 507 stateful business workflows 20 times across 12 LLM models. In a 121,680‑trial ablation, 79,853 attempts failed the executable checks, with 67.24 % terminating cleanly yet still producing wrong field values (77.61 %), unintended extra effects (43.30 %), or missing required effects (25.36 %). The results expose a large consistency gap between tool‑call correctness and actual backend state, urging engineers to validate agents against real data stores rather than only LLM outputs.
⚡ Key Takeaways
- 507 workflows × 20 runs each were evaluated, totaling 121,680 valid trials.
- The benchmark runs agents in isolated MCP tool sessions and checks terminal backend state via executable checks.
- 67.24 % of failed attempts still terminated cleanly, but 77.61 % had wrong field values, highlighting a hidden failure mode.
- The benchmark is available on Hugging Face and can be executed through the OpenEnv framework.
- Executable checks focus on a single field (e.g., ticket status) to determine pass/fail, as illustrated in `sandbox_external_retail_group1.py:test_case_ST003_006`.
Engineers can no longer rely on surface‑level tool‑call success; validating the actual database state is essential for trustworthy agents in production systems.
✅ Practical Steps
- Clone the ThinkingBox repository from Hugging Face and install the OpenEnv environment as described in the paper.
- Run the provided benchmark script `sandbox_external_retail_group1.py:test_case_ST003_006` to reproduce the single‑field executable check.
- Analyze the pass@1 and 20/20 scores to assess your agent’s consistency across repeated runs.
Want the full story? Read the original article.
Read on Hugging Face Blog ↗