---
title: "Comment by Kartik N V J K on \"Semantic Search Over Transcripts in Python: What Embeddings Fix and What They Don't\" | Hashnode"
description: "Defining the correct answers before running the queries is the step most retrieval demos skip, and it is the only reason your \"vector won one, lost one\" result"
url: "https://hashnode.com/posts/semantic-search-over-transcripts-in-python/6ac3469f8ffc43b6b8fbc33a/comment/6ac40da3a88d44ea74845865"
---

[Discussion](https://hashnode.com/posts/semantic-search-over-transcripts-in-python/6ac3469f8ffc43b6b8fbc33a)

Comment [Semantic Search Over Transcripts in Python: What Embeddings Fix and What They Don't](https://hashnode.com/posts/semantic-search-over-transcripts-in-python/6ac3469f8ffc43b6b8fbc33a)

[Kartik N V J K](https://hashnode.com/@kartik-nvj)

[Oct 5](https://hashnode.com/posts/semantic-search-over-transcripts-in-python/6ac3469f8ffc43b6b8fbc33a/comment/6ac40da3a88d44ea74845865)

Defining the correct answers before running the queries is the step most retrieval demos skip, and it is the only reason your "vector won one, lost one" result means anything. The honest LSA-on-28-paragraphs caveat matters too, since fitting on the corpus you are testing flatters recall in a way a pretrained encoder would not. I went through the same measure-against-a-gold-set exercise with LlamaIndex and wrote up how I score it here: [kartiknvjk.hashnode.dev/evaluating-llamaindex-rag…](https://kartiknvjk.hashnode.dev/evaluating-llamaindex-rag-2026)

[Libraryminds](https://hashnode.com/@libraryminds)

[Oct 6](https://hashnode.com/posts/semantic-search-over-transcripts-in-python/6ac3469f8ffc43b6b8fbc33a/comment/6ac40da3a88d44ea74845865#reply-6ac44f48fb220b19d35e1f8c)

I also agree with the point in your article about scoring retrieval separately from the answer, since that is what this test does too.
