---
title: "Discussion on \"I built a lead-intent pipeline on 1.75M posts/day for €10/day. It got 1 trial\" | Hashnode"
description: "Discussion on \"I built a lead-intent pipeline on 1.75M posts/day for €10/day. It got 1 trial\". Post-mortem of a solo SaaS: a self-hosted LLM pipeline finding buying intent in 1.75M social posts a day for €10/day, and why 800 cold emails got 1 trial."
url: "https://hashnode.com/posts/i-built-a-lead-intent-pipeline-on-1-75m-posts-day-for-10-day-it-got-1-trial/6ab0ed44d2b8849594949d89"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [I built a lead-intent pipeline on 1.75M posts/day for €10/day. It got 1 trial](https://vivian-oliveres.hashnode.dev/i-built-a-lead-intent-pipeline-on-1-75m-posts-day-for-10-day-it-got-1-trial?utm_source=hashnode&utm_medium=feed)

Post-mortem of a solo SaaS: a self-hosted LLM pipeline finding buying intent in 1.75M social posts a day for €10/day, and why 800 cold emails got 1 trial.

Sep 21 · 27 min read · vivian-oliveres.hashnode.dev  
Read the article

-   [#machine-learning](https://hashnode.com/tag/machine-learning)
-   [#llm](https://hashnode.com/tag/llm)
-   [#saas](https://hashnode.com/tag/saas)
-   [#indie-hacker](https://hashnode.com/tag/indie-hacker)

## Written by

[Vivian Oliveres](https://hashnode.com/@vivian-oliveres)

@vivian-oliveres · Champagne-sur-seine, France

  
[View profile](https://hashnode.com/@vivian-oliveres)

Sharing thoughts as a Solo Developer

Software engineer (Backend/Data/ML) of ~20 years of experience. Currently building an MVP. Documenting the mess: code, infra, decisions and (lots of) mistakes :-p Learning in public.

## Responses 6

Markdown supported · ⌘↵ to post

[Mihai\_LeanZero](https://hashnode.com/@mihaileanzero)

[Sep 21](https://hashnode.com/posts/i-built-a-lead-intent-pipeline-on-1-75m-posts-day-for-10-day-it-got-1-trial/6ab0ed44d2b8849594949d89/comment/6ab175bf3d86a69656b2863e)

The nightly precision audit is the part I'd want to stress-test hardest: it calls Opus fresh each night to re-judge delivered leads, while the recall benchmark runs off labels Opus produced once and froze. Was that nightly call pinned to a specific Opus checkpoint the whole seven months, or the latest alias? If Anthropic moved the model under it at any point, the audit's meaning shifts even though nothing in the pipeline changed -- and with 339k lines nobody read, that audit was the only signal anyone had that anything upstream was still doing its job.

[Vivian Oliveres](https://hashnode.com/@vivian-oliveres)

[Sep 22](https://hashnode.com/posts/i-built-a-lead-intent-pipeline-on-1-75m-posts-day-for-10-day-it-got-1-trial/6ab0ed44d2b8849594949d89/comment/6ab175bf3d86a69656b2863e#reply-6ab1e771478ca67a90c02abd)

Great question, and I didn't cover it in the article.

Across major versions, the Opus version was deliberate and dated. Every Opus bump is an explicit version in the code with a date: Opus 4.6 in March, then 4.7, 4.8 in May and 5 in July. Labels land in a per-annotator path (Hive-style S3 partitions: `against=opus-4.8/`, `against=opus-5/`), and mixing two partitions was forbidden by rule (defined and used by the ML agent, see the "Building It With AI" section). Also, an Opus version bump had to pass a Cohen's κ check on a reference set first (a measure of how much two labellers agree). For instance κ(Opus 4.8, Opus 5) = 0.813 for n=120, against a 0.80 floor (on my frozen data set). That pass is also why I kept the frozen recall benchmark rather than rebuilding it, so those 477 pairs are still Opus-4.8-labelled. Note that was a judgement call, not a measurement.

But within a major version, you're right, and I had no protection. I used the short alias, not a dated snapshot, so a checkpoint moving under claude-opus-4-8 would have been invisible to me.

How do you handle it on your side? Any unattended agent calling a model has the same problem: dated snapshots, or accept the drift and check the output?

[Mihai\_LeanZero](https://hashnode.com/@mihaileanzero)

[Sep 23](https://hashnode.com/posts/i-built-a-lead-intent-pipeline-on-1-75m-posts-day-for-10-day-it-got-1-trial/6ab0ed44d2b8849594949d89/comment/6ab175bf3d86a69656b2863e#reply-6ab44ce97dfcd2a0b1cf3121)

Mostly the second, accept the drift and rely on catching it downstream. Most of what we call doesn't expose dated snapshots the way Opus apparently does for you, so we're stuck with whatever's current behind a given API or tool. What actually catches a behavior change isn't the model ID, it's whatever the agent's output has to pass afterward, tests, a review gate, a check against known-good output. It's a weaker guarantee than your kappa-gated version bump against a frozen reference set, and reading your setup is honestly making me want to add a pinned-snapshot option anywhere the API actually offers one instead of defaulting to whatever's current.

[Mihai\_LeanZero](https://hashnode.com/@mihaileanzero)

[Sep 21](https://hashnode.com/posts/i-built-a-lead-intent-pipeline-on-1-75m-posts-day-for-10-day-it-got-1-trial/6ab0ed44d2b8849594949d89/comment/6ab0edbea700f015289832e2)

This is one of the most honest build-in-public postmortems I've read here. The retry table stood out to me, the line that anything outside GPU/LLM/network steps gets no retry because a real bug has to fail loudly, not hide behind one. That's the exact rule teams usually lose once retries start creeping in everywhere. We hit something adjacent running unattended agents on macOS launchd: once a step gets retry logic that makes a failure recoverable, the alert rule that used to fire on it quietly stops meaning what it meant when it was written, and nobody goes back to re-tune it once the retry proves itself over a few weeks, so you end up paging on things that now self-heal and staying quiet on the ones that actually changed shape. Did you revisit your 26 alert rules as the vLLM restart-and-resume logic matured, or did the alert set stay fixed from when it was first written? Also curious about the decisions log, once it grows past a few dozen entries does the agent actually read all of it every session, or did you need retrieval or summarization to keep it inside context before it started getting ignored again the same way the stale config did?

[Vivian Oliveres](https://hashnode.com/@vivian-oliveres)

[Sep 21](https://hashnode.com/posts/i-built-a-lead-intent-pipeline-on-1-75m-posts-day-for-10-day-it-got-1-trial/6ab0ed44d2b8849594949d89/comment/6ab0edbea700f015289832e2#reply-6ab0f15682255ab6e6a148cb)

Thanks. Both questions hit real things.

On the alerts: the retry logic and the alert never drifted apart, because the alert doesn't watch steps. Every phase writes a row to a Postgres table (start, end, status), and the rule counts only the latest attempt per (run\_date, phase, slot). A phase that fails and then succeeds on retry is invisible. Only a phase whose last attempt failed pages me. So a step becoming recoverable changes nothing in the alert's meaning.

On the decision log: it never fits in context. 381 entries, ~16,000 lines. A pre-commit hook generates an index, one line per decision with its file and line number. The agent greps the index for the topic, then reads only the entries it points at. Full retrieval. And because the hook regenerates it on every commit, the index can't go stale the way handwritten config does.

PS: Reading ~5,000 words in two minutes is impressive. If there's an LLM in the loop, tell it the answers were in the "Watching It Run" section 🙂

[Mihai\_LeanZero](https://hashnode.com/@mihaileanzero)

[Sep 24](https://hashnode.com/posts/i-built-a-lead-intent-pipeline-on-1-75m-posts-day-for-10-day-it-got-1-trial/6ab0ed44d2b8849594949d89/comment/6ab0edbea700f015289832e2#reply-6ab52c132609c18f088b49c2)

No LLM shortcut on this end, read the whole thing, "Watching It Run" is exactly where both answers were sitting, fair catch. Keying the alert on the latest attempt per (run\_date, phase, slot) rather than watching the step directly is the cleaner design, a retry becoming recoverable just can't touch it. And the pre-commit-generated index for the decision log solves the exact problem we've hit elsewhere, past a few dozen entries an agent either re-reads everything or skims and misses the one line that mattered, a regenerated index that can't go stale is a better answer than either.