---
title: "Comment by anil verma on \"Deep Dive of the Distributed Job Scheduler that powers over 2 Billion daily jobs at PhonePe\" | Hashnode"
description: "Snehasish Roy I appreciate the detailed explanations. I have a question about the job extractor: what is the necessity of running it on each instance? Is it com"
url: "https://hashnode.com/posts/deep-dive-of-the-distributed-job-scheduler-that-powers-over-2-billion-daily-jobs-at-phonepe/65d1086c135bdc53dbd20e4f/comment/65d1dfa50e54ec858f6b4c04"
---

[Discussion](https://hashnode.com/posts/deep-dive-of-the-distributed-job-scheduler-that-powers-over-2-billion-daily-jobs-at-phonepe/65d1086c135bdc53dbd20e4f)

Comment [Deep Dive of the Distributed Job Scheduler that powers over 2 Billion daily jobs at PhonePe](https://hashnode.com/posts/deep-dive-of-the-distributed-job-scheduler-that-powers-over-2-billion-daily-jobs-at-phonepe/65d1086c135bdc53dbd20e4f)

[anil verma](https://hashnode.com/@vermanil)

[Feb 18, 2024](https://hashnode.com/posts/deep-dive-of-the-distributed-job-scheduler-that-powers-over-2-billion-daily-jobs-at-phonepe/65d1086c135bdc53dbd20e4f/comment/65d1dfa50e54ec858f6b4c04)

[Snehasish Roy](https://hashnode.com/@snehasishroy) I appreciate the detailed explanations.

I have a question about the job extractor: what is the necessity of running it on each instance? Is it computationally intensive due to massive record? If we were to run it as a single job only, would we eliminate the need for the leader election component?

I'm curious about the challenges that drive the decision to go with this approach

[Snehasish Roy](https://hashnode.com/@snehasishroy)

[Feb 18, 2024](https://hashnode.com/posts/deep-dive-of-the-distributed-job-scheduler-that-powers-over-2-billion-daily-jobs-at-phonepe/65d1086c135bdc53dbd20e4f/comment/65d1dfa50e54ec858f6b4c04#reply-65d1e803d3d8be9e2568d74d)

> If we were to run it as a single job only, would we eliminate the need for the leader election component?

Yes, but we would limit our throughput (the number of records fetched by an HBase scan in a given period) - If we run multiple concurrent Hbase scans, each without any overlap, then we will be able to scale our scans horizontally and serve the ever increasing traffic :)
