Probe DNS and agent well-knowns before you scrape

Agents waste money when they hit the wrong host, miss /.well-known/ discovery files, or treat a parked domain like a live shop. What to check Apex vs www vs Workers custom domain Whether llms.txt, ro