Semantic Search Over Transcripts in Python: What Embeddings Fix and What They Don't

Thanks Ahmet, this is a good way to split it. I built your version and added it to the post as an update, with the code and the real output. The right answers stay fixed for all three queries, and the script prints which query words the model used.

"reduce churn": only "reduce" is used, so keyword and LSA both find 0 of 3. "reduce cancellations": both words are used. Both find 1 of 3, but LSA ranks the right paragraph first and keyword ranks it second. "subscriber attrition": no word is in the transcript, so both return nothing.

So in this small test, vocabulary changes the result the most, and the learned projection changed the order once and the count never. I have not run the pretrained half yet, so the post says "not run" and gives no numbers for it. If you run it on your side, I would like to compare.

I also added a line near the top that says the vector search in this post is a small LSA stand-in, so the title is not read as a claim about all embeddings.

Thanks for running that comparison and keeping the pretrained half explicitly unmeasured. The printed query vocabulary now makes the zero-result cases explainable, and the unchanged hit count with a different rank is a useful distinction.

I have not run the pretrained version, so I do not have results to compare. For that extension, I would preserve the same corpus, relevance labels and cutoff, and report ranking changes separately from newly retrieved relevant paragraphs. The zero-vector case should remain explicit rather than producing arbitrary nearest neighbours.