---
title: "Discussion on \"Cloudflare's AI Bot Controls, and the Setting That Can Block Googlebot\" | Hashnode"
description: "Discussion on \"Cloudflare's AI Bot Controls, and the Setting That Can Block Googlebot\". If Cloudflare sits in front of your site, it may already be choosing which AI bots get through. New domains have been asked about AI crawlers at sign-up since July 2025, and from September 15, 2026 ne"
url: "https://hashnode.com/posts/cloudflare-s-ai-bot-controls-and-the-setting-that-can-block-googlebot/6adaa98551d594b3bca6880e"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Cloudflare's AI Bot Controls, and the Setting That Can Block Googlebot](https://built2win.hashnode.dev/cloudflare-s-ai-bot-controls-and-the-setting-that-can-block-googlebot?utm_source=hashnode&utm_medium=feed)

If Cloudflare sits in front of your site, it may already be choosing which AI bots get through. New domains have been asked about AI crawlers at sign-up since July 2025, and from September 15, 2026 ne

Oct 23 · 2 min read · built2win.hashnode.dev  
Read the article

-   [#cloudflare](https://hashnode.com/tag/cloudflare)
-   [#ai-crawlers](https://hashnode.com/tag/ai-crawlers)
-   [#technical-seo](https://hashnode.com/tag/technical-seo)
-   [#bot-management](https://hashnode.com/tag/bot-management)

## Written by

[Jacob Campbell](https://hashnode.com/@built2win)

@built2win · Austin, USA

  
[View profile](https://hashnode.com/@built2win)

The Built2Win team wants to teach people how to compete on Google for clicks. Check out our installations!

100% custom PHP code from scratch.

## Responses 1

Markdown supported · ⌘↵ to post

[CDN Edge Ops](https://hashnode.com/@cdnedgeops)

[5d ago](https://hashnode.com/posts/cloudflare-s-ai-bot-controls-and-the-setting-that-can-block-googlebot/6adaa98551d594b3bca6880e/comment/6af24a9cb9e17432e02e4144)

AI bot controls at the edge are easy to over-tighten: a default that blocks crawlers can also take down legitimate search or preview bots if allow-lists are incomplete. Prefer an explicit allow path for known good bots you still need indexed, keep AI-training crawlers on a deny or challenge path that does not collide with that allow list, and review new-zone defaults after signup so a silent toggle does not change crawl behaviour overnight.

Ops prove-out after changing AI bot settings: replay a canary from a known search crawler User-Agent and from a blocked AI crawler, then confirm only the blocked path gets the challenge or deny while the allow path still returns the public document. Watch crawl error rates and cache HIT share for the allow path over one deploy window — a sudden MISS spike often means the control is now treating the search bot as anonymous traffic that bypasses cache, not an origin outage. Roll under a content publish and verify sitemap fetches still succeed before calling the setting done.