---
title: "Discussion on \"Rate Limiting Algorithms: How to Protect Your API Without Annoying Your Users\" | Hashnode"
description: "Discussion on \"Rate Limiting Algorithms: How to Protect Your API Without Annoying Your Users\". How fixed windows, sliding windows, token buckets and leaky buckets work, where each one breaks, and how to pick the right one for your API."
url: "https://hashnode.com/posts/rate-limiting-algorithms-how-to-protect-your-api-without-annoying-your-users-hdp4e/6adf13f827c3f1dc1e9bccfc"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Rate Limiting Algorithms: How to Protect Your API Without Annoying Your Users](https://blog.zyvop.com/rate-limiting-algorithms-how-to-protect-your-api-without-annoying-your-users-hdp4e?utm_source=hashnode&utm_medium=feed)

How fixed windows, sliding windows, token buckets and leaky buckets work, where each one breaks, and how to pick the right one for your API.

Oct 26 · 11 min read · blog.zyvop.com  
Read the article

-   [#apidesign](https://hashnode.com/tag/apidesign)
-   [#backendengineering](https://hashnode.com/tag/backendengineering)
-   [#ratelimiting](https://hashnode.com/tag/ratelimiting)
-   [#redis](https://hashnode.com/tag/redis)
-   [#systemdesign](https://hashnode.com/tag/systemdesign)

## Written by

[ZyVOP](https://hashnode.com/@zyvop)

@zyvop · Delhi, India

  
[View profile](https://hashnode.com/@zyvop)

The Developer Publishing Hub

One platform to write, cross-post to Dev.to, Medium, WordPress, Hashnode, and Bluesky, and protect your SEO with canonical links. Zero paywalls and full content ownership.

## Responses 10

Markdown supported · ⌘↵ to post

[melonape](https://hashnode.com/@melonape)

[Oct 27](https://hashnode.com/posts/rate-limiting-algorithms-how-to-protect-your-api-without-annoying-your-users-hdp4e/6adf13f827c3f1dc1e9bccfc/comment/6ae087e273c73827adacde05)

One more thing for the IP section: on IPv6, key on the /64, not the full address. One machine usually gets a whole /64 and can rotate through addresses for free, so per-address limits barely slow it down.

Mobile carriers are the opposite problem. A lot of people sit behind carrier-grade NAT, so in some regions one IPv4 can be a whole neighborhood. Per-IP limits only really make sense as a loose outer layer.

[ZyVOP](https://hashnode.com/@zyvop)

[Oct 27](https://hashnode.com/posts/rate-limiting-algorithms-how-to-protect-your-api-without-annoying-your-users-hdp4e/6adf13f827c3f1dc1e9bccfc/comment/6ae087e273c73827adacde05#reply-6ae08be3dd51a132862d202b)

Great points! IPv6 address rotation makes per-address limits much easier to bypass, so aggregating requests by a /64 prefix is a useful additional safeguard. On the other hand, carrier-grade NAT shows why strict IP-based limits can unfairly affect legitimate users sharing the same IPv4 address.

I’d treat IP-based throttling as one layer in a broader strategy, combining it with account-level limits, API keys where applicable, and endpoint-specific quotas. The challenge is balancing abuse prevention with avoiding false positives for legitimate users. Thanks for highlighting both sides!

[Crownforge](https://hashnode.com/@thecrownforge)

[Oct 27](https://hashnode.com/posts/rate-limiting-algorithms-how-to-protect-your-api-without-annoying-your-users-hdp4e/6adf13f827c3f1dc1e9bccfc/comment/6ae077ed375e3174907aa8bd)

For variable request costs, I would distinguish an empty bucket from a request that costs more than the bucket can ever hold. Waiting and retrying can fix the first case but not the second, so the client needs a different explanation

[ZyVOP](https://hashnode.com/@zyvop)

[Oct 27](https://hashnode.com/posts/rate-limiting-algorithms-how-to-protect-your-api-without-annoying-your-users-hdp4e/6adf13f827c3f1dc1e9bccfc/comment/6ae077ed375e3174907aa8bd#reply-6ae08bfaf3b6bfa1e8940619)

Absolutely. These are two different failure cases and should be handled differently. An empty bucket means the request may succeed after enough tokens are replenished, so a `429 Too Many Requests` with a meaningful `Retry-After` can make sense. But if the request cost exceeds the bucket’s maximum capacity, waiting will never help. That should be rejected explicitly, with a clear explanation that the request exceeds the allowed limit rather than encouraging pointless retries. Great distinction to include in the implementation and testing strategy!

[Ziwei Yao](https://hashnode.com/@vitayao)

[Oct 26](https://hashnode.com/posts/rate-limiting-algorithms-how-to-protect-your-api-without-annoying-your-users-hdp4e/6adf13f827c3f1dc1e9bccfc/comment/6adf1d55d3e0a4968c437d8d)

One extra layer I’d add for long-running LLM requests is an in-flight concurrency limit. A token bucket caps arrivals, but if calls become much slower, the number running at once can still grow while the arrival rate stays within budget. For usage that isn’t known up front, reserving an upper bound and reconciling with actual usage also needs its own accounting. I’d test a slowdown and a client disconnect separately: the client going away doesn’t necessarily mean the provider stopped doing work.

[ZyVOP](https://hashnode.com/@zyvop)

[Oct 27](https://hashnode.com/posts/rate-limiting-algorithms-how-to-protect-your-api-without-annoying-your-users-hdp4e/6adf13f827c3f1dc1e9bccfc/comment/6adf1d55d3e0a4968c437d8d#reply-6ae00f1f1dbf0ab83c1361ac)

Exactly. This is why an explicit `unknown` state is important in the reservation lifecycle. A client disconnect should not automatically release reserved budget when the provider’s outcome is still unknown. The reservation should remain unresolved until final usage is reconciled or the case is flagged for review, with retries accounting for that outstanding reservation. Otherwise, a transient disconnect could turn into a double-spend.

I’d also make this a separate failure scenario in the test suite: the client disconnects, the provider continues processing, and a retry arrives before the original request’s outcome is known. Great catch—this makes the accounting model much more robust!

[Ziwei Yao](https://hashnode.com/@vitayao)

[Oct 27](https://hashnode.com/posts/rate-limiting-algorithms-how-to-protect-your-api-without-annoying-your-users-hdp4e/6adf13f827c3f1dc1e9bccfc/comment/6adf1d55d3e0a4968c437d8d#reply-6ae060d47651a675141b4b00)

If reconciliation can be retried, I’d also run it twice for the same request and check that the reservation settles only once. That would cover duplicate settlement as well as the retry race.

[ZyVOP](https://hashnode.com/@zyvop)

[Oct 27](https://hashnode.com/posts/rate-limiting-algorithms-how-to-protect-your-api-without-annoying-your-users-hdp4e/6adf13f827c3f1dc1e9bccfc/comment/6adf1d55d3e0a4968c437d8d#reply-6ae08b9ddd51a132862d1b0b)

Agreed. Reconciliation should be idempotent, not just retryable. I’d run it twice for the same request and verify that the reservation is settled only once, with no duplicate budget release or usage charge. An atomic state transition and a unique settlement key can help enforce that guarantee. This catches duplicate settlement in addition to the retry race. Great addition to the test checklist!
