I Added llms.txt to a Next.js Site: A Developer's Guide to Taming AI Crawlers

Something strange happened on one of my client sites this year. The server logs showed a flood of bot traffic, but it was not Googlebot. It was GPTBot, ClaudeBot, CCBot, and a handful of other crawler