@rviole
Nothing here yet.
Nothing here yet.
Mar 10, 2025 · 10 min read · 🚀 Implementation of ADAM In earlier posts, we implemented a momentum-based optimizer, which offered several advantages: Momentum helps avoid getting stuck in local minima, much like in physics 🏃♂️. It smooths updates by acting as a moving averag...
Join discussion
Feb 27, 2025 · 3 min read · Implementing Momentum-based Optimizer 🚀 Context Last time, we implemented a simple learning rate scheduler and identified a major flaw: over time, it becomes unreliable for optimizing the training process. So, we decided to upgrade it. I had always ...
Join discussion
Feb 25, 2025 · 3 min read · 🚀 Learning Rate Scheduling: Fixing the Convergence Issue Context 📚 In my first post, I implemented simple Gradient Descent, and while it worked, we faced a common issue—the convergence of parameters was inconsistent. The intercept (b) converged muc...
Join discussion
Feb 23, 2025 · 2 min read · Exploring Derivatives in Practice Context 📚 After completing a Linear Algebra course, I transitioned to studying Calculus. Unexpectedly, at work, I found myself contemplating derivatives and integrals. This sparked an idea—why not apply derivatives ...
Join discussion