--- title: "Discussion on \"Why AI Applications Need Background Workflows\" | Hashnode" description: "Discussion on \"Why AI Applications Need Background Workflows\". Learn why long running AI tasks need background workflows and how asynchronous processing, retries, and AI agents improve reliability and scalability." url: "https://hashnode.com/posts/why-ai-applications-need-background-workflows/6a9ef29efc4d623b5e92e991" --- [Feed](https://hashnode.com/feed) / Discussion # [Why AI Applications Need Background Workflows](https://axnsh.hashnode.dev/why-ai-applications-need-background-workflows?utm_source=hashnode&utm_medium=feed) Learn why long running AI tasks need background workflows and how asynchronous processing, retries, and AI agents improve reliability and scalability. Sep 7 · 11 min read · axnsh.hashnode.dev Read the article - [#ai](https://hashnode.com/tag/ai) - [#asynchronous-programming](https://hashnode.com/tag/asynchronous-programming) - [#background-workflows](https://hashnode.com/tag/background-workflows) - [#chaiaurcode](https://hashnode.com/tag/chaiaurcode) - [#ai-agents](https://hashnode.com/tag/ai-agents) ## Written by [Aansh Malhotra](https://hashnode.com/@axnsh) @axnsh · Amritsar, India [View profile](https://hashnode.com/@axnsh) GenAI Engineer | Full Stack Developer | Building Scalable AI-Powered Web Apps | ## Responses 6 Markdown supported · ⌘↵ to post [Salman Parvez | ML Systems](https://hashnode.com/@ml-systems) [Sep 7](https://hashnode.com/posts/why-ai-applications-need-background-workflows/6a9ef29efc4d623b5e92e991/comment/6a9f363df3354fce1f6974f7) Per-step retries are the right unit, and they pull a second requirement in behind them: each step has to leave a record of what it produced and from which input, or a retry can silently redo work against a newer version of the document than the step before it saw. We run intake this way for a house — assessor harvest, facade vision, sketch reconciliation, compression — and the thing that made retries safe wasn't the queue, it was giving every step's output a source and a version, so a re-run that disagrees with the earlier run shows up as a conflict instead of overwriting it. [Aansh Malhotra](https://hashnode.com/@axnsh) [Sep 9](https://hashnode.com/posts/why-ai-applications-need-background-workflows/6a9ef29efc4d623b5e92e991/comment/6a9f363df3354fce1f6974f7#reply-6aa19b6adf05e503c4981932) That’s a really good point. Retries are only truly safe when each step knows exactly what input it worked on and what output it produced. Versioning those outputs also makes conflicts visible instead of letting a retry overwrite newer data. Definitely something worth considering for production workflows. [Salman Parvez | ML Systems](https://hashnode.com/@ml-systems) [Sep 14](https://hashnode.com/posts/why-ai-applications-need-background-workflows/6a9ef29efc4d623b5e92e991/comment/6a9f363df3354fce1f6974f7#reply-6aa7d270fbb0d91d69fd1377) Thanks, Aansh. The one rule that made it hold: a re-run never writes over the earlier output, it writes a second claim beside it with its own source and version. Whether the two agree is decided by a separate reconciliation step, not by the retry, and if they disagree the row is marked conflict and nothing downstream reads it until a person resolves it. Retries stay cheap because they can never be destructive, which is what makes it safe to set them aggressively. [Ahmet Özel](https://hashnode.com/@ahmetozel) [Sep 7](https://hashnode.com/posts/why-ai-applications-need-background-workflows/6a9ef29efc4d623b5e92e991/comment/6a9f12aca8d0b315f8c68fd7) The argument gets stronger if you add why the request-response version fails badly rather than gracefully. A long AI call inside an HTTP request does not just make the user wait - it holds memory for the whole duration, so under load you do not get a slow service, you get an OOM kill. Every in-flight request is sitting on a context, and the failure arrives all at once. Two things worth pairing with the workflow queue. Idempotency keys, because retries are the entire point of a durable workflow and an agent step that charges something or sends an email twice is worse than one that failed. And a bound on queue depth with early rejection, since accepting work you cannot finish turns a capacity problem into an outage - a fast 503 is friendlier than a job that sits for twenty minutes and times out. Separating queue wait from execution time in your metrics is the other habit that pays off: they demand opposite fixes, and a single p95 blends them into something you cannot act on. [Aansh Malhotra](https://hashnode.com/@axnsh) [Sep 9](https://hashnode.com/posts/why-ai-applications-need-background-workflows/6a9ef29efc4d623b5e92e991/comment/6a9f12aca8d0b315f8c68fd7#reply-6aa19b2bcb56e36849738b5a) Absolutely agree. The memory/scale point is something I should have emphasized more a long-running request can become a real problem once multiple requests are running at the same time. Idempotency and queue limits are also great points, especially for workflows that can trigger side effects. And separating queue time from execution time in metrics makes a lot of sense for knowing where the actual bottleneck is. Thanks for adding these points! [Ahmet Özel](https://hashnode.com/@ahmetozel) [Sep 11](https://hashnode.com/posts/why-ai-applications-need-background-workflows/6a9ef29efc4d623b5e92e991/comment/6a9f12aca8d0b315f8c68fd7#reply-6aa39e60744aab33bf296c49) One concrete version of the memory point, if you ever expand it: the number to watch is concurrent in-flight requests multiplied by average context size, not request rate. Rate looks healthy right up until it does not, because duration is what multiplies it. Moving the work to a queue is what converts that unbounded product into a worker count you actually chose.