---
title: "Discussion on \"When Retrying Makes Things Worse\" | Hashnode"
description: "Discussion on \"When Retrying Makes Things Worse\". What I learned about timeouts, idempotency, and cascading failures while exploring distributed systems."
url: "https://hashnode.com/posts/when-retrying-makes-things-worse/6ad0f3ce2b150690994ead52"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [When Retrying Makes Things Worse](https://rajatrai.hashnode.dev/when-retrying-makes-things-worse?utm_source=hashnode&utm_medium=feed)

What I learned about timeouts, idempotency, and cascading failures while exploring distributed systems.

Oct 15 · 17 min read · rajatrai.hashnode.dev  
Read the article

-   [#sre](https://hashnode.com/tag/sre)
-   [#distributed-system](https://hashnode.com/tag/distributed-system)
-   [#cloud](https://hashnode.com/tag/cloud)
-   [#guardrails](https://hashnode.com/tag/guardrails)

## Written by

[Rajat Rai](https://hashnode.com/@itsrajatrai)

@itsrajatrai · Remote

  
[View profile](https://hashnode.com/@itsrajatrai)

An engineer who loves to learn and implement things.

👋 Hey, I am Rajat! Working as a Software Engineer at RedHat.

## Responses 3

Markdown supported · ⌘↵ to post

[Mike Dabydeen](https://hashnode.com/@firelinks)

[Oct 16](https://hashnode.com/posts/when-retrying-makes-things-worse/6ad0f3ce2b150690994ead52/comment/6ad208525f054845d61f479f)

Your 3 x 3 x 3 example points at the one guardrail your list doesn't cover. Each technique on it works per call, so every layer can follow all six and C still sees 27 attempts. The fix sits between the layers. The SRE book chapter you cite says to avoid retrying at multiple levels, because attempts multiply by the count at each layer, and suggests a process-wide retry budget, for example 60 retries per minute, after which you fail the request instead of retrying.

In your A, B, C setup that could mean only B retries C, and when B gives up it returns a specific overloaded status that tells A not to retry. Your simulation would show the difference well: add a counter for attempts reaching C, run it once with retries at every layer and once with retries at B only.

[Rajat Rai](https://hashnode.com/@itsrajatrai)

[Oct 16](https://hashnode.com/posts/when-retrying-makes-things-worse/6ad0f3ce2b150690994ead52/comment/6ad208525f054845d61f479f#reply-6ad20a27d8a731026ee812fc)

This is a really good catch. The distinction between per-call retry limits and controlling retries across layers is important, and I should have covered it. Thanks for pointing this out! I'll add that.

[Rajat Rai](https://hashnode.com/@itsrajatrai)

[Oct 16](https://hashnode.com/posts/when-retrying-makes-things-worse/6ad0f3ce2b150690994ead52/comment/6ad208525f054845d61f479f#reply-6ad21a02ff87dbfeba1591a3)

I've updated the article with a section on coordinating retries across layers and added a small simulation comparing retries at every layer with retries at just one layer.