Comment by Ahmet Özel on "Embeddings, Vector Search & RAG from Scratch with Node.js and PostgreSQL (pgvector)"

Ahmet Özel

AI Engineer. Computer Vision, RAG and LLM agents.

The simple chunker has an edge case worth adding to its known limitations: with overlap set to zero, current.slice(-overlap) becomes slice(0), which copies the entire previous chunk. A caller trying to disable overlap can therefore get growing duplicated content instead.

I would handle zero explicitly and validate that overlap is non-negative and smaller than chunkSize. A tiny fixture with several short paragraphs can assert the exact chunk boundaries for zero and positive overlap. That makes the from-scratch implementation more useful for your suggested 300, 800 and 2000-character experiments, where an unnoticed duplication bug would distort the retrieval comparison.

Asfaq Leeon

Senior Software Engineer

Oct 17

Wow, great catch! Also, thanks for reading the blog and trying the code. You’re right: with overlap = 0, current.slice(-overlap) becomes slice(0), so the old snippet copied the whole previous chunk into the next one.

That zero case is easy to miss because positive overlap still appears to work. An explicit zero branch makes the intended behavior much easier to reason about than relying on negative slicing.

I would also keep a progress invariant in the fixture: each emitted chunk must advance to new source text, and the final chunk must terminate without repeating the same suffix forever. Testing empty paragraphs and a paragraph longer than chunkSize alongside overlap=0 would help separate the duplication fix from the chunker's remaining size-limit behavior.

Search Hashnode