3. Gradient Descent: Exploring Momentum

Implementing Momentum-based Optimizer 🚀 Context Last time, we implemented a simple learning rate scheduler and identified a major flaw: over time, it becomes unreliable for optimizing the training process. So, we decided to upgrade it. I had always ...