Our few-shot examples came from the eval set. The 0.94 was fiction.
TL;DR. Our ticket-routing eval scored 0.94 for five weeks. The number was manufactured. We had built a dynamic few-shot selector that retrieved the eight nearest labeled examples for each input, and w
ethanwalkerwrites.hashnode.dev16 min read