---
title: "Discussion on \"A Software Engineer's Intuition for Gradient Descent, Part 2: The Step Size\" | Hashnode"
description: "Discussion on \"A Software Engineer's Intuition for Gradient Descent, Part 2: The Step Size\". Why one learning rate fails on ill-conditioned losses, and how per-parameter step sizes, Newton, and Adam fix it. An intuitive guide with the math."
url: "https://hashnode.com/posts/a-software-engineer-s-intuition-for-gradient-descent-part-2-the-step-size/6ab4a7d7ceed9af2a2f2c6bf"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [A Software Engineer's Intuition for Gradient Descent, Part 2: The Step Size](https://intuitiveml.hashnode.dev/a-software-engineer-s-intuition-for-gradient-descent-part-2-the-step-size?utm_source=hashnode&utm_medium=feed)

Why one learning rate fails on ill-conditioned losses, and how per-parameter step sizes, Newton, and Adam fix it. An intuitive guide with the math.

Sep 24 · 12 min read · intuitiveml.hashnode.dev  
Read the article

-   [#gradient-descent](https://hashnode.com/tag/gradient-descent)
-   [#machine-learning](https://hashnode.com/tag/machine-learning)

## Written by

[Wei Rui](https://hashnode.com/@weirui)

@weirui · San Francisco, USA

  
[View profile](https://hashnode.com/@weirui)

AI Platform Engineer

## Responses

Markdown supported · ⌘↵ to post

No responses yet.