Hamed Karimi wins ECML PKDD Test-of-Time Award
Amazon senior applied scientist Hamed Karimi and coauthors received the 2026 ECML PKDD Test‑of‑Time Award for their 2016 paper “Linear convergence of gradient and proximal‑gradient methods under the Polyak–Łojasiewicz condition,” which revived the 1963 Polyak‑Łojasiewicz inequality to prove linear convergence of gradient‑based training even in non‑convex settings. The work unified a fragmented body of optimization research and gave a theoretical foundation for why modern over‑parameterized models, such as large language models, train efficiently despite having many equally good solutions. Engineers can now reference the PL condition when analyzing or designing new training algorithms, bridging the gap between practice and theory.
⚡ Key Takeaways
- The award recognized a 2016 paper that showed linear convergence under the Polyak–Łojasiewicz condition.
- The Polyak–Łojasiewicz condition (a 1963 inequality) is the key analytical tool for proving fast convergence of gradient and proximal‑gradient methods in non‑convex landscapes.
- The result addresses the performance gap for models with flat valley floors, typical of over‑parameterized systems where traditional strong convexity does not hold.
- Researchers can apply the PL condition to analyze new training methods, using it as a theoretical guarantee for linear convergence.
- The approach assumes the PL inequality holds for the loss landscape; verifying this condition is a prerequisite before applying the convergence guarantees.
Providing a concrete convergence guarantee for gradient‑based training in realistic, non‑convex settings lets ML engineers design and debug optimizers with provable speed‑up, reducing reliance on empirical trial‑and‑error.
✅ Practical Steps
- When evaluating a new optimizer, check whether the loss function satisfies the Polyak–Łojasiewicz condition to leverage linear convergence guarantees.
- Use the PL‑based analysis framework from the 2016 paper to formally bound the convergence rate of gradient or proximal‑gradient methods on your model.
Want the full story? Read the original article.
Read on Amazon Science ↗