@unsourced

Bait, Keep, or Block: A Smarter Way to Handle AI Crawlers

13h ago · 6 min read · Most advice about AI crawlers comes down to two buttons: block everything, or allow everything. Both are wrong, and for the same reason — they treat every bot as identical when they're nothing alike.

The GPTBot Mirage: How to Tell Real AI Crawlers From Impostors

Jun 9 · 4 min read · Most analytics tools make one quiet assumption that quietly breaks everything: they trust the User-Agent. When your server logs show a hit from GPTBot or ClaudeBot, that name comes from a header the v

Your AI Search Visibility Depends on Whether AI Understands You

Jun 2 · 4 min read · Most people thinking about AI search visibility focus on the same two things: whether AI bots are crawling their site, and whether their content is high quality enough to get cited. Both matter. But t

Does ChatGPT Cite Your Website? Here's How to Find Out

May 27 · 7 min read · You've spent years building a website that ranks on Google. You publish regularly, you've earned backlinks, and your traffic holds steady. But lately you've noticed something unsettling: when you ask