---
title: "#deduplication articles | Hashnode"
description: "Explore 11 articles tagged #deduplication on Hashnode. Tutorials, guides, and how-tos from the community."
url: "https://hashnode.com/tag/deduplication"
---

Tag

# #deduplication

11 posts · 0 followers

[Internals Decoded](https://hashnode.com/@internalsdecoded) · [Nov 10](https://hashnode.com/posts/entity-resolution/6af3236ad0c1e8db1237a14d)

### [Entity Resolution: One Thing, Many Names](https://internals-decoded.hashnode.dev/entity-resolution?utm_source=hashnode&utm_medium=feed)

Sam K., Samuel Kim, sam@acme.com. Deciding what's the same thing is harder and more important than it looks.

14 min read · [internals-decoded.hashnode.dev](https://internals-decoded.hashnode.dev/entity-resolution?utm_source=hashnode&utm_medium=feed)

[f0d0r](https://hashnode.com/@f0d0r) · [Sep 24](https://hashnode.com/posts/how-to-organize-40gb-of-ebooks/6ab56e6244cfd0075e2a01b2)

### [How to organize 40GB of ebooks](https://f0d0r.hashnode.dev/how-to-organize-40gb-of-ebooks?utm_source=hashnode&utm_medium=feed)

How to deduplicate and organize a 40GB e-book library using Go, SQLite, MinHash, and LSH. Fast and memory-efficient homelab solution.

14 min read · [f0d0r.hashnode.dev](https://f0d0r.hashnode.dev/how-to-organize-40gb-of-ebooks?utm_source=hashnode&utm_medium=feed)

[Businessware Technologies](https://hashnode.com/@businesswaretech) · [Jul 24](https://hashnode.com/posts/why-search-first-ai-pipelines-work-better-than-browser-agens-for-public-resume-data-extraction/6a6349261f1ae0fffc229cc3)

### [Why Search-First AI Pipelines Work Better Than Browser Agens for Public Resume Data Extraction](https://businesswaretech.hashnode.dev/why-search-first-ai-pipelines-work-better-than-browser-agens-for-public-resume-data-extraction?utm_source=hashnode&utm_medium=feed)

Learn how a search-first AI pipeline extracts, normalizes, deduplicates, and ranks public resume data more efficiently than browser agents.

10 min read · [businesswaretech.hashnode.dev](https://businesswaretech.hashnode.dev/why-search-first-ai-pipelines-work-better-than-browser-agens-for-public-resume-data-extraction?utm_source=hashnode&utm_medium=feed)

[SapotaCorp](https://hashnode.com/@sapotacorpvn) · [Jul 12](https://hashnode.com/posts/survivorship-in-marketing-cloud-which-record-wins-when-a-customer-is-duplicated/6a5382133b88498a315ea183)

### [Survivorship in Marketing Cloud: Which Record Wins When a Customer Is Duplicated](https://sapotacorpvn.hashnode.dev/survivorship-in-marketing-cloud-which-record-wins-when-a-customer-is-duplicated?utm_source=hashnode&utm_medium=feed)

The same person can land in your data as three records, each right about one field and stale about another. Survivorship is the rule that picks one golden record per customer. Here is how the customer type hierarchy decides the winner, and

7 min read · [sapotacorpvn.hashnode.dev](https://sapotacorpvn.hashnode.dev/survivorship-in-marketing-cloud-which-record-wins-when-a-customer-is-duplicated?utm_source=hashnode&utm_medium=feed)

[Yasir Jafri](https://hashnode.com/@yasir323) · [Jun 27](https://hashnode.com/posts/idempotency-key-pattern-python/6a3f89df8a1145f015b7f993)

### [Tap, Tap, Charged Twice: The Idempotency-Key Pattern](https://yasir323.hashnode.dev/idempotency-key-pattern-python?utm_source=hashnode&utm_medium=feed)

How to make unsafe API requests safe to retry with the failure modes the happy path hides.

13 min read · [yasir323.hashnode.dev](https://yasir323.hashnode.dev/idempotency-key-pattern-python?utm_source=hashnode&utm_medium=feed)

[Ankit Raj](https://hashnode.com/@ankitsrivastava) · [Mar 16](https://hashnode.com/posts/fix-duplicate-rows-in-sql-merge-using-row-number/69b8498a2ad6ae5184dbfb64)

### [Fix Duplicate Rows in SQL MERGE Using ROW\_NUMBER](https://ankit-data-engineering.hashnode.dev/fix-duplicate-rows-in-sql-merge-using-row-number?utm_source=hashnode&utm_medium=feed)

Why frequent ingestion pipelines create duplicate rows and the simple SQL pattern that prevents them.

5 min read · [ankit-data-engineering.hashnode.dev](https://ankit-data-engineering.hashnode.dev/fix-duplicate-rows-in-sql-merge-using-row-number?utm_source=hashnode&utm_medium=feed)

[Tech Insights Hub](https://hashnode.com/@JohnMikes) · [Feb 12](https://hashnode.com/posts/borg-backup-deduplicating-archiver/698dd7cb77dd7207c5309d27)

### [Borg Backup: Deduplicating Archiver](https://topperblog.hashnode.dev/borg-backup-deduplicating-archiver?utm_source=hashnode&utm_medium=feed)

Borg Backup: Deduplicating Archiver The Problem That Changed Everything Modern development requires modern solutions. Table of Contents Why This Matters Quick Start Implementation Best Practices FAQ Conclusion Why This Matters Essential for product...

2 min read · [topperblog.hashnode.dev](https://topperblog.hashnode.dev/borg-backup-deduplicating-archiver?utm_source=hashnode&utm_medium=feed)

[Surya Sathi](https://hashnode.com/@surya-sathi) · [Jan 26](https://hashnode.com/posts/idempotency/697709f61c5ccfad21f2f449)

### [Idempotency](https://systems-notes.hashnode.dev/idempotency?utm_source=hashnode&utm_medium=feed)

Making Systems Correct Under Retries and Concurrency

10 min read · [systems-notes.hashnode.dev](https://systems-notes.hashnode.dev/idempotency?utm_source=hashnode&utm_medium=feed)

[Sivaprasad Murali](https://hashnode.com/@crystallinecore) · [Jun 16, 2025](https://hashnode.com/posts/how-i-built-fylex-a-fast-python-tool-to-clean-organize-and-de-duplicate-files/684fcb10c20e024965987662)

### [How I Built Fylex: A Fast Python Tool to Clean, Organize, and De-Duplicate Files](https://fylex.hashnode.dev/how-i-built-fylex-a-fast-python-tool-to-clean-organize-and-de-duplicate-files?utm_source=hashnode&utm_medium=feed)

Supercharge your file management with Fylex — a blazing-fast, Python-powered tool for deduplicating, copying, and organizing files with ease.

3 min read · [fylex.hashnode.dev](https://fylex.hashnode.dev/how-i-built-fylex-a-fast-python-tool-to-clean-organize-and-de-duplicate-files?utm_source=hashnode&utm_medium=feed)

[Akhil Kondapaneni](https://hashnode.com/@akhil012) · [Oct 7, 2023](https://hashnode.com/posts/deduplication-demystified-navigating-duplicates-in-event-driven-systems/6520cea1a8c0701e13da781f)

### [Deduplication Demystified: Navigating duplicates in Event-Driven Systems](https://akhil-kondapaneni.hashnode.dev/deduplication-demystified-navigating-duplicates-in-event-driven-systems?utm_source=hashnode&utm_medium=feed)

Introduction When working with streaming queues such as Google Cloud Pub/Sub or Apache Kafka, ensuring data integrity is paramount. These platforms provide "at least once" delivery semantics, meaning that a message may be delivered to the consumer mo...

5 min read · [akhil-kondapaneni.hashnode.dev](https://akhil-kondapaneni.hashnode.dev/deduplication-demystified-navigating-duplicates-in-event-driven-systems?utm_source=hashnode&utm_medium=feed)

[1](https://hashnode.com/posts/deduplication-demystified-navigating-duplicates-in-event-driven-systems/6520cea1a8c0701e13da781f)
