We Followed 173 Million Domains for 8 Years. ~40M Died.
Key takeaways
We built an 8-year reachability panel of 172,959,928 registered domains from 80 monthly Common Crawl archives (January 2018 to May 2026) — every domain Common Crawl ever fetched a robot
crawlora.hashnode.dev12 min read