Snehasish Roy I appreciate the detailed explanations.
I have a question about the job extractor: what is the necessity of running it on each instance? Is it computationally intensive due to massive record? If we were to run it as a single job only, would we eliminate the need for the leader election component?
I'm curious about the challenges that drive the decision to go with this approach