EPEric Pughindep4b.hashnode.dev·Feb 13, 2025 · 7 min readThe Four Horsemen of the Judging ApocalypseWant the tl;dr;? Jump to the bottom for some tables the suggest which style of judgements you need! Unveiling the Judging Spectrum: From Perfection to Bias There are various types of Judgements, and recently, I've been contemplating the emergence of...00
EPEric Pughindep4b.hashnode.dev·Jan 23, 2025 · 3 min readCan a click through rate ever be more than 100%?Recently while leveraging some UBI data for search quality evaluation we ran into the issue of having a single search lead to multiple click throughs. Imagine you click on a result, then go back, and then click again on the same result! What to do? I...00
EPEric Pughindep4b.hashnode.dev·Dec 20, 2024 · 3 min readAll I want for Solr 10 is...Of course it starts with an upgrade to the latest Lucene…. I’m looking forward to a 10.1 version of Lucene with all the early adopter bugs worked out ;-). The rest of the list, in no specific order: The MVP of the new Admin UI! Something Christos ha...01D
EPEric Pughindep4b.hashnode.dev·Dec 20, 2024 · 4 min readImplicit Judgements and COECDriving the evaluation aspect of the Evaluation Framework is the choice of how we model user clicks to decide what is a good search result and what isn’t. We are starting with Clicks over Expected Clicks (COEC). This is a well understood approach for...00
EPEric Pughindep4b.hashnode.dev·Dec 12, 2024 · 2 min readThe Dev console is NOT the place for debugging OpenSearch Queries!!!Coming out of a week of deep pairing with my colleagues Daniel Wrigley and Jeff Zemerick on the upcoming Search Quality Evaluation Framework for OpenSearch, one thing that I’ve noticed after some intense swarming sessions is that we abuse the Dev Con...00
