After 10 Heads in a Row, Small LLMs Bet Up to 82% on an 11th
Small language models exhibit a strong streak bias, betting up to 82% on an 11th coin flip after observing ten consecutive heads, even when explicitly told the coin is fair and flips are independent. This reveals that simple prompting only trims the bias by a few points, indicating persistent probabilistic miscalibration in small LLMs. Engineers should account for such biases when deploying LLMs in decision‑making pipelines.
⚡ Key Takeaways
- Small LLMs assign up to 82% probability to the 11th flip being heads after ten heads in a row.
- Prompting the model with “the coin is fair and flips are independent” reduces the bias only marginally.
- The bias persists despite instruction tuning, highlighting a trade‑off between model size and probabilistic reasoning accuracy.
Understanding and mitigating streak bias is crucial for reliable LLM‑driven predictions in applications like risk assessment or automated decision systems.
✅ Practical Steps
- Apply the concepts from this article to your own system design.
Want the full story? Read the original article.
Read on Towards Data Science ↗