---
title: "Sravan Vidiyala (@sravanvidiyala) | Hashnode"
description: "Engineering LLM systems that hold in production. Read the latest articles by Sravan Vidiyala on Hashnode."
url: "https://hashnode.com/@sravanvidiyala"
---

@sravanvidiyala

# Sravan Vidiyala

@sravanvidiyala · Sydney, Australia · Joined June 2026

Engineering LLM systems that hold in production.

## Comments

[Manual. The tool wasn't released yet, so there was no production to get a signal from. I harvested 13,474 real tool descriptions, ran the detector over them of which twenty servers failed. All twenty were wrong. Nothing prompted it. Nothing looked broken. I went looking because I'd written both the tests and the rules, so the tests passing didn't tell me anything. That's the part worth generalising I suppose, that this cateogry of bug has no symptom. You don't catch it by watching, you catch it by getting data you didn't make up. I only half-fixed it, mind. My "12 of 13 attack shapes" uses a list I wrote myself, so it has the same problem. That one's still open.](https://hashnode.com/posts/my-test-suite-was-green-my-detector-was-wrong/6a796091a1f0931c50030008/comment/6a7a0cfcdd81a7d4712b5533#reply-6a7a5cfd0505268b37c6be83)

Reply · Article · Aug 10 · [My test suite was green. My detector was wrong.](https://hashnode.com/posts/my-test-suite-was-green-my-detector-was-wrong/6a796091a1f0931c50030008)
