---
title: "Discussion on \"Why Detecting AI-Generated Text Is Harder Than You Think (And What I Built Anyway)\" | Hashnode"
description: "Discussion on \"Why Detecting AI-Generated Text Is Harder Than You Think (And What I Built Anyway)\". Building a tool that admits \"I don't know\" instead of faking 99% accuracy."
url: "https://hashnode.com/posts/why-detecting-ai-generated-text-is-harder-than-you-think-and-what-i-built-anyway/6ab490aa913d7f6b467755b7"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Why Detecting AI-Generated Text Is Harder Than You Think (And What I Built Anyway)](https://detectaihashnodedev.hashnode.dev/why-detecting-ai-generated-text-is-harder-than-you-think-and-what-i-built-anyway?utm_source=hashnode&utm_medium=feed)

Building a tool that admits "I don't know" instead of faking 99% accuracy.

Sep 24 · 2 min read · detectaihashnodedev.hashnode.dev  
Read the article

## Written by

[chen jiayan](https://hashnode.com/@chenjiayan)

@chenjiayan · New York, USA

  
[View profile](https://hashnode.com/@chenjiayan)

Indie hacker exploring AI watermarking & content provenance.

Building AI tools to detect what machines create.

## Responses 1

Markdown supported · ⌘↵ to post

[AI Tools Primer](https://hashnode.com/@aitoolsprimer)

[Sep 24](https://hashnode.com/posts/why-detecting-ai-generated-text-is-harder-than-you-think-and-what-i-built-anyway/6ab490aa913d7f6b467755b7/comment/6ab59e1ca86588f46f06dae1)

The perplexity point deserves a warning label of its own. The 2023 Stanford study (Liang et al., "GPT detectors are biased against non-native English writers") ran seven detectors over TOEFL essays and got an average false-positive rate of 61.22%, while essays by US eighth graders were classified almost perfectly. Their explanation is the signal you list first: the flagged essays had significantly lower perplexity, because plainer vocabulary reads as "smooth".

A 2026 re-test (Al Ali, Helcl and Libovický) found the English rate down to 23.1%, and no systematic bias against non-native writers of Czech, so the effect seems to depend on the language. Still roughly one essay in four in English.

So per-length calibration is a good start. I'd add a line to the verdict whenever low perplexity is doing most of the work, saying plainly that second-language English triggers it too. Turnitin's own guidance is to use its score "to initiate a conversation, not to draw a conclusion", and showing which signals fired is the part that makes that conversation possible.