The vector-prefilter-then-Jev split is a clean design and the cost table makes the case well. One thing I'd watch for once this runs in production: you're caching resolved phrase-to-token mappings permanently for cost, but your own first lesson was that most of Jev's misses came down to incomplete catalog descriptions, and fixing those improved the whole system's accuracy. A permanent cache keyed only on the phrase doesn't know the description behind a past resolution has since been corrected, so a wrong answer learned before a description fix can sit in the cache indefinitely and invisibly, while every fresh, uncached query benefits from the fix right away. Keying the cache to a description version, even just a hash of the description text, and invalidating on catalog edits would close that gap without giving up the cost win.
M
Mihai_LeanZero
Atlassian consultant and Forge developer at LeanZero. We do Data Center to Cloud migrations and build Forge apps, and I write up what I learn at leanzero.net.
Thanks, this is a sharp catch, and you're right. Keying the resolution cache only on the phrase means a description fix silently fails to reach anything already cached, which cuts against the whole point that better catalog descriptions are what moved our accuracy. We'll address this in the next release: hashing the description text (a catalog version) into the cache key and invalidating on catalog edits, so we keep the cost win without letting a pre-fix answer linger. Appreciate you flagging it.
The vector-prefilter-then-Jev split is a clean design and the cost table makes the case well. One thing I'd watch for once this runs in production: you're caching resolved phrase-to-token mappings permanently for cost, but your own first lesson was that most of Jev's misses came down to incomplete catalog descriptions, and fixing those improved the whole system's accuracy. A permanent cache keyed only on the phrase doesn't know the description behind a past resolution has since been corrected, so a wrong answer learned before a description fix can sit in the cache indefinitely and invisibly, while every fresh, uncached query benefits from the fix right away. Keying the cache to a description version, even just a hash of the description text, and invalidating on catalog edits would close that gap without giving up the cost win.