Optimizers explained

Mar 28, 2023 · 6 min read · This article summarizes optimization algorithms using unified notation and explains the ideas behind each improvement. Basic Gradient Descent Types ❓ How do we find optimal parameters for a network?​ Suppose a model has parameters \(\theta \in \mathb...