Tag feed

#evals

26 posts0 followers

Explore Hashnode

Alternatives

Trending tags this week

ACAlexander Codesalexandercodes.hashnode.dev29m ago · 10 min read

How to Evaluate AI-Generated SQL on Databricks

An AI assistant can produce a query that runs and returns a plausible result. But how do we know if it's correct? Suppose a user asks: Which enterprise customers had revenue growth last quarter? An

0

EBElisé Baraka M.ebmurha.hashnode.devMay 22 · 11 min read

Building a RAG Engine — Part 4: Engine Controls

The previous part redesigned the entire system into a domain-neutral engine structure with reference storage. Graph extraction was one of the planned capabilities for the engine: for a start, basic kn

0

NANishant Agarwalthisisnish.hashnode.devMay 6 · 8 min read

I Built a Self-Improving Stock Screener to Understand How LLMs Actually Reason

Why I Built This It started with a mundane problem: I wanted a better way to filter stocks. Not trading signals, not price alerts — just a structured monthly process to surface equities worth looking

0

DADavid Aronchickdistributedthoughts.orgMay 5 · 8 min read

The Loop Is Only as Good as the Metric

On Thursday I wrote about Karpathy's autoresearch, the 630-line training loop that runs 100 ML experiments overnight on a single GPU while you sleep. The post generated a lot of conversation, and most of it centered on the automation: agents doing re...

0

JBJordan Bourbonnaisclawpulse.hashnode.devApr 27 · 3 min read

A Practical Guide to Monitoring AI Agents in Production: Beyond Traditional LLM Evaluation

What you'll learn The critical difference between monitoring and evaluation in production AI systems How to build a real-time observability stack for AI agents Practical strategies for catching failures before they impact users How to integrate moni...

0

RSRahul Sehrawatai-zero-to-hero.hashnode.devApr 13 · 11 min read

The Evals-First Loop: Write the Test Before the Prompt

I can tell, within about two minutes of joining a new team, whether they're serious about building with LLMs. I don't have to look at the code. I don't have to see the prompts. I ask one question: "Show me your eval." A team that's serious says "here...

0

DGDan Girelliniengineering.fractional.aiFeb 12 · 8 min read

Your evals have a Rotten Tomatoes problem

You push a change to a prompt and your eval score drops from 0.91 to 0.84. Something got worse, but the score doesn’t tell you what. So you start re-running the pipeline, tweaking your inputs, pouring

0

AAniblog.anirudha.devNov 16, 2025 · 14 min read

Building AI Tools You Can Trust

You want to build an AI application. Something useful. Something your users can rely on. But here's the problem: How do you know it's actually good? You can build an app that generates summaries, answers questions, or writes emails. It works. Your te...

0

BKBen Kadishengineering.fractional.aiNov 4, 2025 · 2 min read

On the new larger context windows

Today the latest LLMs have large context windows up to ~1 million tokens. There are many occasions when this larger context window can be useful: Context engineering: injecting rich system/user conte

0

AAniblog.anirudha.devOct 26, 2025 · 5 min read

Unit Tests for Intelligence

A few months ago, as I was exploring machine learning while working on a project, one of my models kept behaving in a weird way. I had built a classifier to detect cats in images. During training, accuracy was awesome, near 99%. But in production, it...

0

#evals

Search Hashnode

#evals

Explore Hashnode

Trending tags this week

How to Evaluate AI-Generated SQL on Databricks

Building a RAG Engine — Part 4: Engine Controls

I Built a Self-Improving Stock Screener to Understand How LLMs Actually Reason

The Loop Is Only as Good as the Metric

A Practical Guide to Monitoring AI Agents in Production: Beyond Traditional LLM Evaluation

The Evals-First Loop: Write the Test Before the Prompt

Your evals have a Rotten Tomatoes problem

Building AI Tools You Can Trust

On the new larger context windows

Unit Tests for Intelligence