dep4b.hashnode.devThe Four Horsemen of the Judging ApocalypseWant the tl;dr;? Jump to the bottom for some tables the suggest which style of judgements you need! Unveiling the Judging Spectrum: From Perfection to Bias There are various types of Judgements, and recently, I've been contemplating the emergence of...Feb 13, 2025·7 min read
dep4b.hashnode.devCan a click through rate ever be more than 100%?Recently while leveraging some UBI data for search quality evaluation we ran into the issue of having a single search lead to multiple click throughs. Imagine you click on a result, then go back, and then click again on the same result! What to do? I...Jan 23, 2025·3 min read
dep4b.hashnode.devAll I want for Solr 10 is...Of course it starts with an upgrade to the latest Lucene…. I’m looking forward to a 10.1 version of Lucene with all the early adopter bugs worked out ;-). The rest of the list, in no specific order: The MVP of the new Admin UI! Something Christos ha...Dec 20, 2024·3 min read
dep4b.hashnode.devImplicit Judgements and COECDriving the evaluation aspect of the Evaluation Framework is the choice of how we model user clicks to decide what is a good search result and what isn’t. We are starting with Clicks over Expected Clicks (COEC). This is a well understood approach for...Dec 20, 2024·4 min read
dep4b.hashnode.devThe Dev console is NOT the place for debugging OpenSearch Queries!!!Coming out of a week of deep pairing with my colleagues Daniel Wrigley and Jeff Zemerick on the upcoming Search Quality Evaluation Framework for OpenSearch, one thing that I’ve noticed after some intense swarming sessions is that we abuse the Dev Con...Dec 12, 2024·2 min read
