yeah we hit this exact thing once we scaled up workers, per process politeness means nothing when the whole fleet hammers the same account or IP at once so we ended up building a shared limiter scoped the way the provider actually counts it. retries need their own cap too, we keep them around 10 to 20 percent of active requests or a throttle event turns into a retry storm that's worse than the original spike. also don't trust status codes blindly, github throttles with a 403 and a zero remaining header instead of the usual 429, and reset headers are epoch seconds for some providers but seconds until reset for others, so you basically have to classify this stuff per provider. one thing that saved us a lot of pain, if retry after is longer than your job deadline just reschedule the whole job instead of parking a worker asleep, and only retry the non idempotent calls when you've got idempotency keys backing them
yeah we hit this exact thing once we scaled up workers, per process politeness means nothing when the whole fleet hammers the same account or IP at once so we ended up building a shared limiter scoped the way the provider actually counts it. retries need their own cap too, we keep them around 10 to 20 percent of active requests or a throttle event turns into a retry storm that's worse than the original spike. also don't trust status codes blindly, github throttles with a 403 and a zero remaining header instead of the usual 429, and reset headers are epoch seconds for some providers but seconds until reset for others, so you basically have to classify this stuff per provider. one thing that saved us a lot of pain, if retry after is longer than your job deadline just reschedule the whole job instead of parking a worker asleep, and only retry the non idempotent calls when you've got idempotency keys backing them