VVioleinmachine-learning-journey.hashnode.dev·Mar 10, 2025 · 10 min read4. Gradient Descent: Discovering Adam Optimizer🚀 Implementation of ADAM In earlier posts, we implemented a momentum-based optimizer, which offered several advantages: Momentum helps avoid getting stuck in local minima, much like in physics 🏃♂️. It smooths updates by acting as a moving averag...00
VVioleinmachine-learning-journey.hashnode.dev·Feb 27, 2025 · 3 min read3. Gradient Descent: Exploring MomentumImplementing Momentum-based Optimizer 🚀 Context Last time, we implemented a simple learning rate scheduler and identified a major flaw: over time, it becomes unreliable for optimizing the training process. So, we decided to upgrade it. I had always ...00
VVioleinmachine-learning-journey.hashnode.dev·Feb 25, 2025 · 3 min read2. Gradient Descent: The First Step in Optimization🚀 Learning Rate Scheduling: Fixing the Convergence Issue Context 📚 In my first post, I implemented simple Gradient Descent, and while it worked, we faced a common issue—the convergence of parameters was inconsistent. The intercept (b) converged muc...00
VVioleinmachine-learning-journey.hashnode.dev·Feb 23, 2025 · 2 min read1. Gradient Descent: My First Implementation🚀Exploring Derivatives in Practice Context 📚 After completing a Linear Algebra course, I transitioned to studying Calculus. Unexpectedly, at work, I found myself contemplating derivatives and integrals. This sparked an idea—why not apply derivatives ...00
