AI bot controls at the edge are easy to over-tighten: a default that blocks crawlers can also take down legitimate search or preview bots if allow-lists are incomplete. Prefer an explicit allow path for known good bots you still need indexed, keep AI-training crawlers on a deny or challenge path that does not collide with that allow list, and review new-zone defaults after signup so a silent toggle does not change crawl behaviour overnight.

Ops prove-out after changing AI bot settings: replay a canary from a known search crawler User-Agent and from a blocked AI crawler, then confirm only the blocked path gets the challenge or deny while the allow path still returns the public document. Watch crawl error rates and cache HIT share for the allow path over one deploy window — a sudden MISS spike often means the control is now treating the search bot as anonymous traffic that bypasses cache, not an origin outage. Roll under a content publish and verify sitemap fetches still succeed before calling the setting done.