---
title: "Discussion on \"Build a Web Scraping API with FastAPI, Celery & Redis (2026)\" | Hashnode"
description: "Discussion on \"Build a Web Scraping API with FastAPI, Celery & Redis (2026)\". Build a production-grade web scraping API with FastAPI, Celery, and Redis in Python. Schedule scrape jobs, return async results, add rate limiting, and deploy with Docker in 2026."
url: "https://hashnode.com/posts/build-a-web-scraping-api-with-fastapi-celery-redis-2026/6ab6a4aa51a337e585fc2bb1"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Build a Web Scraping API with FastAPI, Celery & Redis (2026)](https://blog.zyvop.com/build-a-web-scraping-api-with-fastapi-celery-redis-2026?utm_source=hashnode&utm_medium=feed)

Build a production-grade web scraping API with FastAPI, Celery, and Redis in Python. Schedule scrape jobs, return async results, add rate limiting, and deploy with Docker in 2026.

Sep 25 · 15 min read · blog.zyvop.com  
Read the article

-   [#background](https://hashnode.com/tag/background)
-   [#fastapiasyncscrapingservice](https://hashnode.com/tag/fastapiasyncscrapingservice)
-   [#scraping](https://hashnode.com/tag/scraping)
-   [#turnscraperintoapipython](https://hashnode.com/tag/turnscraperintoapipython)

## Written by

[ZyVOP](https://hashnode.com/@zyvop)

@zyvop · Delhi, India

  
[View profile](https://hashnode.com/@zyvop)

The Developer Publishing Hub

One platform to write, cross-post to Dev.to, Medium, WordPress, Hashnode, and Bluesky, and protect your SEO with canonical links. Zero paywalls and full content ownership.

## Responses 2

Markdown supported · ⌘↵ to post

[OnlineProxy](https://hashnode.com/@onlineproxyio)

[Sep 30](https://hashnode.com/posts/build-a-web-scraping-api-with-fastapi-celery-redis-2026/6ab6a4aa51a337e585fc2bb1/comment/6abc8f74d0e962e34c7358b5)

Your compose setup won't run as written, cause tasks.py hardcodes localhost for redis instead of reading REDIS\_URL. Also curl\_cffi, playwright and playwright\_stealth are missing from requirements.txt, and there's no playwright install step

The browser task reports success when errors got swallowed, and its max\_retries never retries anything. Bulk drops task\_id, so revoke can't cancel those jobs.

Security is the worst part, since url is a read SSRF with no blocklist and GET /jobs hands out the api\_key column to anyone with no auth or ownership checks

[ZyVOP](https://hashnode.com/@zyvop)

[Sep 30](https://hashnode.com/posts/build-a-web-scraping-api-with-fastapi-celery-redis-2026/6ab6a4aa51a337e585fc2bb1/comment/6abc8f74d0e962e34c7358b5#reply-6abc9843e37c9a0c6448ba92)

Thank you, I will recheck