PUProxy Universeinproxyuniverse.hashnode.dev·1d ago · 4 min readStop Finding Out From a Timeout: Monitor Your Proxy Provider's Uptime in ~40 Lines of PythonSubtitle: After the 2026 outage wave (9Proxy, IP2World, PIA S5), I refuse to learn about provider downtime from a failed job. Here's the tiny health-check monitor that pings me on Telegram before my p00
RRoamProxyinroamproxy.hashnode.dev·1d ago · 4 min readPlaywright + Proxies: Sticky Sessions, Rotation, and the SOCKS5 TrapYou wired a proxy into requests in one line. Then you tried the same thing with Playwright and got a soup of ERR_TUNNEL_CONNECTION_FAILED, mystery CAPTCHAs, and a bandwidth bill that made no sense. Be00
CTChandler Thompsoninchandlerthompson.hashnode.dev·2d ago · 4 min readThe state of web scraping in the AI era: still relevant or a relic?Every few months someone declares web scraping dead. APIs cover more ground than they did, and the models can read a page and hand you structured data, so why keep writing parsers? I had the same thou00
RRoamProxyinroamproxy.hashnode.dev·3d ago · 3 min readWe Sent 10,000 Requests Through a Residential Proxy Network. Here's the Raw Data.Most proxy benchmarks you'll find are vendor marketing: a success rate with no methodology, no failure counts, no raw data. We run a residential proxy network, and we wanted numbers we could actually 00
RRoamProxyinroamproxy.hashnode.dev·5d ago · 4 min readHow to Avoid Getting Blocked While Web Scraping: 7 Rules That Actually MatterMost "how to not get blocked" advice is a list of 30 tips where 25 barely move the needle. After running scrapers in production for a while, here are the seven that account for nearly all of the diffe00
ATAethyn Teaminaethyn-io.hashnode.dev·Jul 12 · 7 min readOne Identity Per Task: Keeping an AI Agent's Browser Session CoherentMost proxy advice was written for scrapers, and if you copy it into an agent you will break your agent in a way that's hard to debug. The advice goes: rotate your IP on every request. For a scraper, t01O
RRoamProxyinroamproxy.hashnode.dev·Jul 12 · 3 min readRotating vs. Sticky Proxies: A Practical Guide for Web ScrapingWhen you route a scraper through a proxy network, one decision quietly determines whether your job succeeds or gets blocked in the first ten minutes: should each request get a fresh IP, or should a ba00
ATAethyn Teaminaethyn-io.hashnode.dev·Jul 10 · 4 min readWhy Your Scraper Works Locally but Dies in Production (and How to Actually Fix It)Same code, same site — green on your laptop, 403 on AWS. It's almost never your code. It's your network. You built a scraper. It runs beautifully on your laptop. You deploy it to AWS / GCP / a VPS, 11O
TWTony Wangincrawlora.hashnode.dev·Jul 10 · 9 min readAI vs Traditional Web Scraping: Which Wins, WhenKey takeaways Traditional scraping (CSS/XPath selectors) is fast, cheap, and near-100% accurate on stable pages — but brittle: by one industry estimate, 10–15% of crawlers need maintenance every week01O
TWTony Wangincrawlora.hashnode.dev·Jul 10 · 11 min readBest SERP APIs in 2026 for Rank Tracking and Search DataKey takeaways Google has no official SERP API — the Custom Search JSON API closed to new customers in 2025 and shuts down January 1, 2027 — so production rank tracking runs on third-party SERP APIs. 00
