---
title: "Discussion on \"Which Malaysian websites block AI crawlers? I checked 100\" | Hashnode"
description: "Discussion on \"Which Malaysian websites block AI crawlers? I checked 100\". First published on yappatyih.com.\nTL;DR\nAn AI crawler block is a rule in a website's robots.txt file that tells one named bot, such as OpenAI's GPTBot, not to fetch the site's pages. On 20 October 202"
url: "https://hashnode.com/posts/which-malaysian-websites-block-ai-crawlers-i-checked-100/6ae3ee42def76554d56b0c6c"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Which Malaysian websites block AI crawlers? I checked 100](https://yappatyih.hashnode.dev/which-malaysian-websites-block-ai-crawlers-i-checked-100?utm_source=hashnode&utm_medium=feed)

First published on yappatyih.com. TL;DR An AI crawler block is a rule in a website's robots.txt file that tells one named bot, such as OpenAI's GPTBot, not to fetch the site's pages. On 20 October 202

Oct 30 · 17 min read · yappatyih.hashnode.dev  
Read the article

-   [#seo](https://hashnode.com/tag/seo)
-   [#ai](https://hashnode.com/tag/ai)
-   [#web-development](https://hashnode.com/tag/web-development)
-   [#malaysia](https://hashnode.com/tag/malaysia)

## Written by

[Yap Pat Yih](https://hashnode.com/@yappatyih)

@yappatyih · Kuala Lumpur, Malaysia

  
[View profile](https://hashnode.com/@yappatyih)

GM and co-founder of BlackRevo. Founder of Habitus Mind. Building Lumen.

I am Yap Pat Yih, General Manager and co-founder of BlackRevo, an SEO and web systems company in Kota Damansara, Selangor, and founder of Habitus Mind. I am building Lumen. A software engineer turned General Manager, I work on systems where people and AI work together. I write about running a company and building with AI at https://yappatyih.com/

## Responses

Markdown supported · ⌘↵ to post

No responses yet.
