Give Claude or Cursor the web: MeshArc MCP in 60 secondsAdd one URL to Claude, Claude Code or Cursor and your assistant reads pages as Markdown, maps sites and reports changes. A real run: 2 credits.9 min read·sidmishra.hashnode.dev14h ago
Best web scraping APIs in 2026, priced per 1,000 successful pagesEight scraping APIs at the ~$100 plan, priced per 1,000 pages you can use: plain, rendered, residential, walled sites, and who bills 404s.10 min read·sidmishra.hashnode.dev1d ago
Crawl a website into BigQuery or Snowflake on a scheduleA scheduled crawler that MERGEs pages into BigQuery or Snowflake, loads only what changed, and never treats a blocked page as deleted.10 min read·sidmishra.hashnode.dev4d ago
Scrape a Cloudflare-protected site for $0.40 per 1,000 pages: the session rungRender the challenge once, then read every other page with the earned session at 2 credits9 min read·sidmishra.hashnode.dev1ONov 6
How to keep a RAG knowledge base fresh without re-embedding everythingHash, compare, delete deliberately: re-embed only the pages that changed, and never let a blocked crawl wipe your index.9 min read·sidmishra.hashnode.devOct 28