Libraryminds

I also agree with the point in your article about scoring retrieval separately from the answer, since that is what this test does too.

Thanks Ahmet, this is a good way to split it. I built your version and added it to the post as an update, with the code and the real output. The right answers stay fixed for all three queries, and the script prints which query words the model used. "reduce churn": only "reduce" is used, so keyword and LSA both find 0 of 3. "reduce cancellations": both words are used. Both find 1 of 3, but LSA ranks the right paragraph first and keyword ranks it second. "subscriber attrition": no word is in the transcript, so both return nothing. So in this small test, vocabulary changes the result the most, and the learned projection changed the order once and the count never. I have not run the pretrained half yet, so the post says "not run" and gives no numbers for it. If you run it on your side, I would like to compare. I also added a line near the top that says the vector search in this post is a small LSA stand-in, so the title is not read as a claim about all embeddings.