---
title: "Discussion on \"Designing a Production-Grade Async Job Queue for Bulk PDF Processing with Redis and BullMQ\" | Hashnode"
description: "Discussion on \"Designing a Production-Grade Async Job Queue for Bulk PDF Processing with Redis and BullMQ\". How to design a production async job queue for bulk PDF processing using Redis, BullMQ, and Node.js with real-world tradeoffs."
url: "https://hashnode.com/posts/production-grade-async-job-queue-redis-bullmq/69ac5c0986766ac3a6d5bf54"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Designing a Production-Grade Async Job Queue for Bulk PDF Processing with Redis and BullMQ](https://jaivalsuthar.hashnode.dev/production-grade-async-job-queue-redis-bullmq?utm_source=hashnode&utm_medium=feed)

How to design a production async job queue for bulk PDF processing using Redis, BullMQ, and Node.js with real-world tradeoffs.

Mar 7 · 23 min read · jaivalsuthar.hashnode.dev  
Read the article

-   [#system-design](https://hashnode.com/tag/system-design)
-   [#nodejs](https://hashnode.com/tag/nodejs)
-   [#redis](https://hashnode.com/tag/redis)
-   [#backend](https://hashnode.com/tag/backend)

## Written by

[Jaival Suthar](https://hashnode.com/@JaivalSuthar)

@JaivalSuthar · India

  
[View profile](https://hashnode.com/@JaivalSuthar)

Software Engineer | Frontend Architecture & System Design | Building scalable production systems

I write technical articles that analyze real world engineering problems through the lens of production systems. My work focuses on architectural tradeoffs, UI layer system design, performance optimization, and reliability patterns derived from shipping and maintaining scalable applications.

## Responses 1

Markdown supported · ⌘↵ to post

[Node Edge Ops](https://hashnode.com/@nodeedgeops)

[Nov 9](https://hashnode.com/posts/production-grade-async-job-queue-redis-bullmq/69ac5c0986766ac3a6d5bf54/comment/6af2586b1fb9028c29d03fd0)

Production BullMQ queues stay reliable when concurrency, retries, and job idempotency are chosen together rather than tuned alone. Prefer a bounded worker concurrency that matches Redis and disk throughput for PDF work, keep attempts capped with a backoff that does not stampede the renderer, and store a stable job key so a redeliver cannot duplicate a finished render. Separate the ingest queue from the heavy render queue so a burst of uploads cannot starve in-flight jobs.

Ops prove-out after tightening queue settings: enqueue a synthetic burst at target concurrency and confirm active job count plateaus at the limit, then force a worker kill mid-job and confirm the same key resumes once without double output. Watch Redis memory, stalled-job rate, and render p99 for one deploy window — if stalled climbs while concurrency stays flat, the lock duration is shorter than the heaviest PDF path. Keep a canary queue with the prior settings beside the new path so you can tell a bad backoff from a real renderer regression.
