Turning Database Grading into RL Rewards: Post-Training Agents with ThinkingBox
The post introduces ThinkingBox, a system that turns executable checks of final database state into reward signals for post‑training agents, and demonstrates its impact by fine‑tuning Qwen3.8‑27B with GRPO for 50 steps. Using an offline policy‑relative difficulty filter to select training tasks, average success on the 507‑task ThinkingBox‑Bench rose from 51.70 % to 60.78 %, and the count of tasks solved on all twenty attempts increased from 38 to 107. This shows that database‑state‑based rewards can substantially improve repeatable correctness for LLM‑driven agents in production‑like environments.
⚡ Key Takeaways
- Average success improved from 51.70 % to 60.78 % after 50 GRPO steps on Qwen3.8‑27B.
- Training pool was built using offline policy‑relative difficulty, selecting tasks whose base‑policy success probability was estimated before training.
- Trade‑off observed: repeatable success (pass@20) increased sharply while at‑least‑once coverage (pass@1) showed a slight decline.
- Integration: run the provided “Run the training stack” script to apply the reward loop and GRPO fine‑tuning.
- Prerequisite: access to ThinkingBox‑Bench tasks formatted for executable database checks; tasks must be pre‑evaluated with the base policy to compute difficulty.
Engineers can leverage concrete database state checks as cheap, deterministic reward signals to RL‑fine‑tune LLM agents, yielding higher reliability on critical transactional workflows without building complex simulators.
✅ Practical Steps
- Clone the ThinkingBox‑Training repository from GitHub and run the “Run the training stack” script to start the GRPO fine‑tuning process.
- Use the offline policy‑relative difficulty estimator on your task set to select a fixed training pool before training begins.
- After fine‑tuning, evaluate the model on ThinkingBox‑Bench by running each of the 507 tasks 20 times to compute pass@1, pass@20, and observed 20/20 metrics.
Want the full story? Read the original article.
Read on Hugging Face Blog ↗