Comment by Kartik N V J K on "Deleting Personal Data from a RAG System in .NET"

CommentDeleting Personal Data from a RAG System in .NET

Kartik N V J K

AI Developer | Making AI reliable, trustworthy & accessible to everyone | Active community contributor

The overnight-retry rewriting the deleted record is the part I see most teams miss: deletion gets treated as a single event, not something that has to hold against every async job that can re-ingest. Building lineage at ingest instead of similarity-searching the vector store for a name later is the right call, since that search never catches all the derived chunks. How do you handle a record that is mid-ingestion when the erasure request lands?

PK

Patrick Kearns

Senior .Net Engineer

2d ago

I think the ingestion would need to be Atomic really. I'd mark the subject as blocked before starting the purge, then have ingestion workers check that marker before processing and again when committing each derived write. Then the final check needs to be atomic with the write where possible otherwise, deletion can land between the check and the insert.

Search Hashnode