---
title: "Comment by Aldo on \"How to Know When ChatGPT Stops Recommending Your Product\" | Hashnode"
description: "The 16-of-32 to 12-of-32 example needs a within-week baseline before I'd call it a real loss. Those 32 answers combine different prompts and engines, and one ge"
url: "https://hashnode.com/posts/how-to-know-when-chatgpt-stops-recommending-your-product/6af5bc2decc4c89b789822ba/comment/6af651d40aa4ca2c7b95ba89"
---

[Discussion](https://hashnode.com/posts/how-to-know-when-chatgpt-stops-recommending-your-product/6af5bc2decc4c89b789822ba)

Comment [How to Know When ChatGPT Stops Recommending Your Product](https://hashnode.com/posts/how-to-know-when-chatgpt-stops-recommending-your-product/6af5bc2decc4c89b789822ba)

[Aldo](https://hashnode.com/@aldoquantic)

[4d ago](https://hashnode.com/posts/how-to-know-when-chatgpt-stops-recommending-your-product/6af5bc2decc4c89b789822ba/comment/6af651d40aa4ca2c7b95ba89)

The 16-of-32 to 12-of-32 example needs a within-week baseline before I'd call it a real loss. Those 32 answers combine different prompts and engines, and one generation per cell can move quite a bit without anything changing on the site.

I'd repeat each fixed prompt a few times in fresh sessions, retain the answers and citations, then compare mention frequency per engine and prompt. Record search mode, locale and the model/version when exposed. Keep “recommended by name”, “linked as a source” and “described accurately” as separate columns; a brand can lose a citation without disappearing from the recommendation.

Also, “why didn't you include us?” is a follow-up generated explanation, not an execution trace of the original answer. It can suggest things to verify, but shouldn't establish the cause of the drop.

For a team building its own monitoring pipeline, QuanticData's AI Visibility API is worth evaluating alongside a manual baseline; I work on QuanticData. I'd still validate automated observations against the particular consumer/search experience the team actually cares about, rather than treating every API answer as identical to that UI.

[AI Search Analyzer](https://hashnode.com/@aisearchanalyzer)

[4d ago](https://hashnode.com/posts/how-to-know-when-chatgpt-stops-recommending-your-product/6af5bc2decc4c89b789822ba/comment/6af651d40aa4ca2c7b95ba89#reply-6af68e070b631f71fc791c34)

Thanks, Aldo. This is a better method than the one in the post, and we agree with all of it. The 16-to-12 example was there to show the arithmetic of the 10-point rule, not to prove a loss, and one generation per cell is too noisy to call it on its own. Repeating each fixed prompt a few times in fresh sessions, and logging search mode, locale and model, is what turns a wobble into a signal. Keeping "recommended by name", "linked as a source" and "described accurately" as separate columns is a great call too, because they move independently. And you're right that the answer to "why didn't you include us?" is a generated explanation, not a log of the model's reasoning. That's why the post says to test each reason against your own site before acting on it.
