Tag feed

#llm-evaluation

14 posts0 followers

Explore Hashnode

Alternatives

Trending tags this week

KNKartik N V J Kkartiknvjk.hashnode.devJun 16 · 9 min read

How I stopped my RAG CI gate from lying to me

I have learned to distrust a fast green check on a RAG eval. The failure has a shape I now recognize. A PR lands late in the day. The RAG eval runs in 38 seconds and goes green. Twelve hours later the

0

SJShola Jegedefreecodecamp.orgJun 13 · 30 min read

Mastra vs LangChain: Building an AI Agent Pipeline and Analyzing the Data

A week ago, I saw this tweet: I had just shipped SupportMesh, a multi-tenant AI support platform built on Mastra, so I had opinions from production. I liked the .dowhile() loop, the typed step schem

0

AHAdam Hillhwaph.hashnode.devJun 12 · 16 min read

It's LLMs All the Way Down

Generative AI (GenAI) systems and Large Language Models (LLMs) are empowering us to tackle new types of problems and enabling the implementation of smart, autonomous (or semi-autonomous) systems. Howe

0

RYRamhee Yeonramieeee.meJun 11 · 5 min read

[Paper Review] Operationalizing Large Language Models for Clinical Research Data Extraction: Methods, Quality Control, and Governance

The paper elaborates methodologies for LLM text extraction from research papers. It is not only giving methodologies for extraction of meaningful data, but also assessment and choosing right models. 1

0

NPNikhil Pareeknikhil-p-blogs.hashnode.devJun 3 · 6 min read

Tool-calling eval is four problems, not one

I want to start with a trace that still bothers me. An agent fails to book a flight. The model called search_flights with departure_date="next Friday". The endpoint expected an ISO date, returned a 40

0

AKAnup Karanjkarwowhow.hashnode.devMay 10 · 11 min read

Your AI Agent Returns HTTP 200 With Confidently Wrong Answers — Fix It

The agent returned a 200. The customer lost $4,200. The monitoring dashboard showed green. This is the failure mode that keeps me up at night — not the one where your AI agent crashes with a stack trace, but the one where it succeeds at the wrong thi...

0

AKAnup Karanjkarwowhow.hashnode.devMay 9 · 11 min read

Your AI Agent Returns HTTP 200 With Confidently Wrong Answers — Fix It

The agent returned a 200. The customer lost $4,200. The monitoring dashboard showed green. This is the failure mode that keeps me up at night — not the one where your AI agent crashes with a stack trace, but the one where it succeeds at the wrong thi...

0

Ssatoruaiknowlejuice.hashnode.devApr 13 · 8 min read

Evaluation, Monitoring, and Model Degradation in Production AI Systems

Last post covered the implementation layer — how speech-to-text, audio emotion, and facial analysis actually run in real-time systems. This one covers what happens after deployment. How you evaluate, monitor, and catch degradation before your users d...

0

CNChris Naughtontensorops.hashnode.devFeb 22 · 2 min read

A New Framework for Detecting LLM Hallucinations in Critical Defense Scenarios

When it comes to deploying large language models in sensitive domains like defense, accuracy isn't just a preference—it's a necessity. That’s why Justin Norman’s release of DoDHaluEval v0.1.0 caught my attention. This open-source framework is specifi...

0

JCJuan Carlos Olamendyjuancolamendy.hashnode.devNov 25, 2025 · 8 min read

The Statistical Reality of LLM Evaluation: What Works, What Doesn't, and When It Matters

Your LLM scored 85% on your test set. How confident are you in that number? What if I told you it might actually be anywhere between 70% and 95%? Most engineering teams ship LLM systems based on evaluation numbers that look precise but hide massive u...

0

#llm-evaluation

Search Hashnode

#llm-evaluation

Explore Hashnode

Trending tags this week

How I stopped my RAG CI gate from lying to me

Mastra vs LangChain: Building an AI Agent Pipeline and Analyzing the Data

It's LLMs All the Way Down

[Paper Review] Operationalizing Large Language Models for Clinical Research Data Extraction: Methods, Quality Control, and Governance

Tool-calling eval is four problems, not one

Your AI Agent Returns HTTP 200 With Confidently Wrong Answers — Fix It

Your AI Agent Returns HTTP 200 With Confidently Wrong Answers — Fix It

Evaluation, Monitoring, and Model Degradation in Production AI Systems

A New Framework for Detecting LLM Hallucinations in Critical Defense Scenarios

The Statistical Reality of LLM Evaluation: What Works, What Doesn't, and When It Matters