---
title: "Discussion on \"I built a free checker: which AI crawlers can actually read your site?\" | Hashnode"
description: "Discussion on \"I built a free checker: which AI crawlers can actually read your site?\". robots.txt tells GPTBot to stay out. Does your server agree? A 10-second test."
url: "https://hashnode.com/posts/i-built-a-free-checker-which-ai-crawlers-can-actually-read-your-site/6ad865dcac05f4550526f045"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [I built a free checker: which AI crawlers can actually read your site?](https://mirelleoskarvy.hashnode.dev/i-built-a-free-checker-which-ai-crawlers-can-actually-read-your-site?utm_source=hashnode&utm_medium=feed)

robots.txt tells GPTBot to stay out. Does your server agree? A 10-second test.

Oct 21 · 3 min read · mirelleoskarvy.hashnode.dev  
Read the article

-   [#ai](https://hashnode.com/tag/ai)
-   [#seo](https://hashnode.com/tag/seo)
-   [#webdev](https://hashnode.com/tag/webdev)
-   [#wordpress](https://hashnode.com/tag/wordpress)

## Written by

[Mirelle Oskarvy](https://hashnode.com/@mirelleoskarvy)

@mirelleoskarvy · Vilnius, Lithuania

  
[View profile](https://hashnode.com/@mirelleoskarvy)

Marketing at MB Skydis — notes on AI agents, product research and publishing

## Responses

Markdown supported · ⌘↵ to post

No responses yet.