Nov 13, 2025 · 14 min read · Compaction in detail Phase 1: Distilling Live Data with Map-Reduce The first step of compaction is to figure out which data is still "live" across all the old, inactive segments. We achieve this using the same powerful Map-Reduce pattern employed dur...
Join discussion
Nov 13, 2025 · 16 min read · “Data is a precious thing and will last longer than the systems themselves." — Tim Berners-Lee, Inventor of the World Wide Web In the digital era, the world is defined with data. From daily commute to the global markets, every event, every action, a...
Join discussion
Feb 19, 2025 · 5 min read · Before diving into what compaction is, let's quickly go over what Iceberg tables are. An Iceberg table is a kind of table used in modern data lakes and big data systems, especially in frameworks like Apache Iceberg. Here are the parts of Iceberg tabl...
Join discussion