ethanwalker

@ethanwritesai

San Francisco, USAJoined May 2026

About

Nothing here yet.

Available for

Nothing here yet.

ethanwalker's blogs

Promptfoo is a CI gate, not an eval framework. Treating it like one cost us $4,200.ethanwalkerwrites.hashnode.dev2 posts

Articles Threads Comments1

Recently published

Eethanwalkerethanwalkerwrites.hashnode.dev

0

Your eval criteria are code. Version them like code.

1d ago · 4 min read · A judge prompt is an implementation. The criterion it encodes is a contract, and ours rotted for three months before anyone noticed. L;DR: We version our prompts and our eval datasets in git. We never

Join discussion

Eethanwalkerethanwalkerwrites.hashnode.dev

0

Promptfoo is a CI gate, not an eval framework. Treating it like one cost us $4,200.

May 26 · 4 min read · What you will learn in this post: Why a passing Promptfoo regression suite does not mean your eval is working How to wire a parallel judge-validation pipeline against production traces The structur

Join discussion

ethanwalker

About

Available for

ethanwalker's blogs

Recently published

Your eval criteria are code. Version them like code.

Promptfoo is a CI gate, not an eval framework. Treating it like one cost us $4,200.

Search Hashnode

ethanwalker

About

Available for

ethanwalker's blogs

Recently published

Your eval criteria are code. Version them like code.

Promptfoo is a CI gate, not an eval framework. Treating it like one cost us $4,200.