Tag feed

#ai-evaluation

8 posts1 followers

Explore Hashnode

Alternatives

VIVictor Isuovictorisuo.hashnode.devJun 27 · 11 min read

Why I Built a Separate Evaluation System Before Trusting My Own AI

The Industrial AI Copilot had been running for three weeks when I realised I had no idea whether it was actually working. Not in the catastrophic sense — no exceptions, no crashes, no hallucinations I

0

FCFrank Chenrespan.hashnode.devApr 28 · 13 min read

Your LLM Judge Has Opinions. They're Not About Quality。

When your eval score goes up, the natural conclusion is that your model got better. But there's another explanation: your LLM judge has systematic biases, and your latest change happened to produce ou

0

Aaiagentmemoryaiagentmemory.hashnode.devApr 7 · 12 min read

LLM Memory Evaluation: Benchmarking Agent Recall and Retention

LLM Memory Evaluation: Benchmarking Agent Recall and Retention LLM memory evaluation systematically assesses how well large language models store, retrieve, and use information over time. It quantifies an AI agent's capacity to retain relevant contex...

0

SMSheila Meyer-Steynthe-ai-learning-journey.hashnode.devJan 16 · 3 min read

How I Used AI to Build Real Job Interview Deliverables

Context This was not a theoretical exercise.I was preparing for a real job interview with real expectations and a limited time. The interview required two concrete deliverables: A solution architecture document An executive-level interview presenta...

0

HAHéctor Arriolahectorarriola.comSep 19, 2025 · 6 min read

AI Evals, Without the Jargon: How We’re Keeping Our Agents Honest at our Agency

When we started rolling out voice and chat agents at Hillflare, it felt like opening a thousand tabs at once. Every new client, every script, every accent, every “quick tweak” to a prompt multiplied the ways things could go right… or sideways. Readin...

0

GDGabi Dobocanblog.telepat.ioNov 22, 2024 · 5 min read

Unlocking the Power of FEET: A New Framework for Evaluating AI Models

Arxiv: https://arxiv.org/abs/2411.01322v1 PDF: https://arxiv.org/pdf/2411.01322v1.pdf Authors: Jeffrey N. Chiang, John Lee, Simon A. Lee Published: 2024-11-02 What Does the FEET Paper Claim? At the heart of the paper "FEET: A Framework For Evaluatin...

0

TXTom X Nguyendwarvesf.hashnode.devOct 16, 2024 · 4 min read

Evaluate Chatbot Agent by User Simulation

When building a chatbot agent, it's important to evaluate its performance and user satisfaction. One effective method is user simulation, which involves creating virtual users to interact with the chatbot and assess its responses. This approach allow...

0

#ai-evaluation

Search Hashnode

#ai-evaluation

Explore Hashnode

Why I Built a Separate Evaluation System Before Trusting My Own AI

Your LLM Judge Has Opinions. They're Not About Quality。

LLM Memory Evaluation: Benchmarking Agent Recall and Retention

How I Used AI to Build Real Job Interview Deliverables

AI Evals, Without the Jargon: How We’re Keeping Our Agents Honest at our Agency

Unlocking the Power of FEET: A New Framework for Evaluating AI Models

Evaluate Chatbot Agent by User Simulation

Trending tags this week