---
title: "Discussion on \"How to run large-scale web scraping jobs with Zenrows Batch\" | Hashnode"
description: "Discussion on \"How to run large-scale web scraping jobs with Zenrows Batch\". Run 100,000 URLs as one managed job: webhooks, HMAC checks, failed-task reruns, CSV input, scheduling and a credit safety cap."
url: "https://hashnode.com/posts/how-to-run-large-scale-web-scraping-jobs-with-zenrows-batch/6ac385d83a7f60ba68616de4"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [How to run large-scale web scraping jobs with Zenrows Batch](https://zenrows.hashnode.dev/how-to-run-large-scale-web-scraping-jobs-with-zenrows-batch?utm_source=hashnode&utm_medium=feed)

Run 100,000 URLs as one managed job: webhooks, HMAC checks, failed-task reruns, CSV input, scheduling and a credit safety cap.

Oct 5 · 8 min read · zenrows.hashnode.dev  
Read the article

-   [#largescalewebscrapingapi](https://hashnode.com/tag/largescalewebscrapingapi)
-   [#zenrows](https://hashnode.com/tag/zenrows)
-   [#zenrows-batch](https://hashnode.com/tag/zenrows-batch)
-   [#batch](https://hashnode.com/tag/batch)
-   [#large-scale-scraping](https://hashnode.com/tag/large-scale-scraping)
-   [#web-scraping-api](https://hashnode.com/tag/web-scraping-api)
-   [#web-scraping](https://hashnode.com/tag/web-scraping)
-   [#python](https://hashnode.com/tag/python)

## Written by

[Oyedele Tioluwani](https://hashnode.com/@Tioluwani)

@Tioluwani · Nigeria

  
[View profile](https://hashnode.com/@Tioluwani)

Developer, Technical Writer

Python developer Studying Mechanical Engineering Machine Learning enthusiast

## Responses 1

Markdown supported · ⌘↵ to post

[OnlineProxy](https://hashnode.com/@onlineproxyio)

[Oct 7](https://hashnode.com/posts/how-to-run-large-scale-web-scraping-jobs-with-zenrows-batch/6ac385d83a7f60ba68616de4/comment/6ac68ff4fd1974a223b2aeb3)

hmac verification looks correct against the docs but test it against real live deliveries. the timestamp never gets validated, so captured payloads could replay if zenrows doesn't check freshness server side

watch loop only treats completed and stopped as terminal, so verify whether failed and deleted need handling before leaving this running. polling ceilings are best effort since in-flight tasks overshoot, so use api key credit caps as your actual backstop with a buffer
