I am planning to implement a delayed job queue system in nodejs(express js) for tasks such as app analytics,image processing,file processing etc.. I tried following libraries
https://github.com/Automattic/kue
https://github.com/rvagg/node-worker-farm
I have following queries
- Do I need to host the job runner redis db on separate server machine
- What happens if the nodejs process restarts(whether task will resume or not)
- How to recover tasks if the server machine crashed and restarted
- How to stop application/database server without interrupting scheduled jobs
Marcus
Full Stack Developer
I would take a look at Agenda. The jobs are stored on mongodb so if your app shuts down and restarts you need to have some kind of graceful shutdown to cancel that particular job. you can set up many different jobs with Agenda and it is relatively easy to set up. i have a job set up and it's only 10 lines.