---
title: "Nagappan S (@imkarthik03) | Hashnode"
description: "Building scalable products at the intersection of Full Stack Engineering & AI. Read the latest articles by Nagappan S on Hashnode."
url: "https://hashnode.com/@imkarthik03"
---

@imkarthik03

# Nagappan S

@imkarthik03 · Chennai, India · Joined September 2026

Building scalable products at the intersection of Full Stack Engineering & AI

[Website](https://nagspidey.co.in/)

## Comments

[Agreed on verbosity bias. One cheap extra check on your human-graded set: correlate the judge's scores with answer length. If that correlation is stronger than the one in the human scores, the bias is showing. The post suggests aiming for roughly 0.85 correlation with human raters and re-checking regularly.](https://hashnode.com/posts/llm-as-a-judge-using-one-model-to-grade-another-and-where-it-fools-you/6aeafeb3f128828a14a16cc1/comment/6aeb3aa66a76c5c99da82800#reply-6af1a5e524e8782be41d2a61)

Reply · Article · Nov 9 · [LLM-as-a-Judge: Using One Model to Grade Another (and Where It Fools You)](https://hashnode.com/posts/llm-as-a-judge-using-one-model-to-grade-another-and-where-it-fools-you/6aeafeb3f128828a14a16cc1)

[Logging the pre-rerank rank of whatever the reranker puts first is a cheap way to tell whether the window is the bottleneck, and recall@50 of the fused list is the number that caps everything after it. The tokenizer point on identifiers like calculate\_tax\_v2 isn't in the post and matters. On weighting the keyword list for identifier-style queries instead of plain k=60: I haven't tested it. The post only says to keep k=60 unless you have labelled relevance data, so I'd try it against a labelled query set before trusting it.](https://hashnode.com/posts/hybrid-search-and-reranking-fixing-retrieval-after-vector-search-falls-short/6ae9d62e3b872791f3339ce9/comment/6aeaa0e2ead8145b951529fe#reply-6af1a5cb4b4391b4b0718317)

Reply · Article · Nov 9 · [Hybrid Search and Reranking: Fixing Retrieval After Vector Search Falls Short](https://hashnode.com/posts/hybrid-search-and-reranking-fixing-retrieval-after-vector-search-falls-short/6ae9d62e3b872791f3339ce9)

[The post doesn't cover the fusion key, so thanks for raising it. I'd pick one unit for both legs, passage level, and carry the parent document ID as metadata so the reranker and citations can still use it. Your test case is the right one: a document the keyword leg returns whose relevant passage only the dense leg returns. Without a consistent key, a high rank in both lists isn't a well-defined claim.](https://hashnode.com/posts/hybrid-search-and-reranking-fixing-retrieval-after-vector-search-falls-short/6ae9d62e3b872791f3339ce9/comment/6aed94c5b2cd1143f171a2cd#reply-6af1a5b7a83903c56a0414a7)

Reply · Article · Nov 9 · [Hybrid Search and Reranking: Fixing Retrieval After Vector Search Falls Short](https://hashnode.com/posts/hybrid-search-and-reranking-fixing-retrieval-after-vector-search-falls-short/6ae9d62e3b872791f3339ce9)

[That's the hierarchical pattern from the post (small children for retrieval, parent section for generation), and it should help: the semantic fragments averaged 43 tokens, which is too thin to answer from, so swapping the section in addresses exactly that. I can't give you a number though. The 69% vs 54% figures come from the FloTorch benchmark, which didn't test that combination. I'd run it on your own corpus and measure end-to-end answer accuracy, not retrieval recall, since Chroma's 91.9% recall for semantic chunking suggests the retrieval side isn't the problem.](https://hashnode.com/posts/why-how-you-split-documents-decides-your-rag-quality/6ae82a163bf65c4425103568/comment/6ae9e767b115302c2bd3f28b#reply-6af1a555ae5498661777725f)

Reply · Article · Nov 9 · [Why How You Split Documents Decides Your RAG Quality](https://hashnode.com/posts/why-how-you-split-documents-decides-your-rag-quality/6ae82a163bf65c4425103568)

[Yes. Tool descriptions get the same treatment as tool output: untrusted until proven otherwise. In practice that means pinning descriptions at approval time and diffing them on every load, so a server can't change its description after you've trusted it. I also show the full description to the user rather than a summary, and keep tools from different servers from sharing a context with high-privilege credentials. The MCPTox numbers are the reason: if the payload sits in the description, the agent has read it before any output filter runs.](https://hashnode.com/posts/prompt-injection-why-ai-agents-shouldn-t-trust-their-own-tools/6ac8900f47ebf6b7c629c6ca/comment/6ac93d032528f23f10d8494b#reply-6acdd359761f328b046ee303)

Reply · Article · Oct 13 · [Prompt Injection: Why AI Agents Shouldn't Trust Their Own Tools](https://hashnode.com/posts/prompt-injection-why-ai-agents-shouldn-t-trust-their-own-tools/6ac8900f47ebf6b7c629c6ca)

[Agreed on both. Versioning the dataset and keeping the previous release's cases as a frozen slice is what separates "the system got worse" from "the coverage got better". I'd report the stable slice and the full set side by side. On trajectories, constraints are the better contract: required effects, forbidden actions and resource limits, rather than one canonical path. That way a better strategy isn't penalised just for taking a different route.](https://hashnode.com/posts/evals-how-do-you-actually-know-your-ai-feature-works/6ac48bb15d0e8169a118170c/comment/6ac53cedbe529719bcce1bb7#reply-6acdd333d57024c379df1982)

Reply · Article · Oct 13 · [Evals: How Do You Actually Know Your AI Feature Works?](https://hashnode.com/posts/evals-how-do-you-actually-know-your-ai-feature-works/6ac48bb15d0e8169a118170c)

[That silent fallback is a nasty one. A parse failure that returns a middle score looks like a result. I'd make the parser fail loudly and track the parse-failure rate as its own metric. For the judge, I'd do both. Re-check it against a small human-labelled set on a fixed cadence, and also whenever the judge model or its prompt changes. Pinning the judge version helps, because then drift only happens when you choose it. A sudden shift in the score distribution with no system change is a good trigger too.](https://hashnode.com/posts/evals-how-do-you-actually-know-your-ai-feature-works/6ac48bb15d0e8169a118170c/comment/6ac6962204eafa6d98700716#reply-6acdd311d57024c379df180c)

Reply · Article · Oct 13 · [Evals: How Do You Actually Know Your AI Feature Works?](https://hashnode.com/posts/evals-how-do-you-actually-know-your-ai-feature-works/6ac48bb15d0e8169a118170c)

[Fair correction. TCP already recovers ordinary packet loss, so the real failure is the connection dropping mid-response. The reconnect test you describe is the right one: cut after a complete SSE event, then check whether the client marks the answer incomplete, restarts, or resumes from an event ID. I'd make resume-by-event-ID the target, since it avoids both a visible glitch and regenerating the whole answer.](https://hashnode.com/posts/why-ai-replies-stream-word-by-word/6ac34ccf409651402523c315/comment/6ac39a2f6dbf382c5678bc73#reply-6acdd2d3e5d5c9e441728b07)

Reply · Article · Oct 13 · [Why AI replies stream word-by-word](https://hashnode.com/posts/why-ai-replies-stream-word-by-word/6ac34ccf409651402523c315)

[Buffering until the stop signal is the safe default. If you need to act earlier, an incremental JSON parser that only commits complete values works, but I'd still never execute the call before the arguments validate. It's not in the series yet, so it's a good candidate for a follow-up post. Thanks for the nudge.](https://hashnode.com/posts/why-ai-replies-stream-word-by-word/6ac34ccf409651402523c315/comment/6ac54d574e3ae971a15544d5#reply-6acdd272acaf114305c1ac7f)

Reply · Article · Oct 13 · [Why AI replies stream word-by-word](https://hashnode.com/posts/why-ai-replies-stream-word-by-word/6ac34ccf409651402523c315)

[You should score that path, and it's where most of the signal is. I'd add dedicated trajectory cases for exactly those scenarios: a mid-task change of request, a tool returning a 429 or timeout, a malformed tool response. What gets scored there isn't the final answer alone. It's the decisions: did it back off and retry a bounded number of times, did it stop and report instead of looping, did it carry the new instruction forward, and did it avoid repeating side-effecting calls. Rule-based checks cover most of it (call counts, retry limits, no duplicate writes), with an LLM judge only for the fuzzier "did it handle the changed ask correctly" part. Clean-path-only evals tend to look great right up until a flaky dependency shows up in production. How are you injecting failures in your setup, mocked tools or real chaos in staging?](https://hashnode.com/posts/evals-how-do-you-actually-know-your-ai-feature-works/6ac48bb15d0e8169a118170c/comment/6ac4920a811880132ddab0ed#reply-6ac49880b009d278956d5789)

Reply · Article · Oct 6 · [Evals: How Do You Actually Know Your AI Feature Works?](https://hashnode.com/posts/evals-how-do-you-actually-know-your-ai-feature-works/6ac48bb15d0e8169a118170c)
