Apr 3 · 5 min read · A streaming data warehouse built on RisingWave and Apache Iceberg combines the freshness of a real-time pipeline with the analytical depth of a traditional data warehouse—without the cost of either a dedicated streaming engine or a commercial warehou...
Join discussion
Apr 3 · 5 min read · Apache Iceberg combined with RisingWave provides the ideal foundation for IoT data lakehouses. RisingWave ingests high-frequency sensor streams via Kafka, applies real-time transformations using materialized views, and sinks enriched data into Iceber...
Join discussion
Mar 29 · 13 min read · TL;DR: Databricks organizes everything in a layered hierarchy — Account → Metastore → Workspace → Catalog → Schema → Table. Understanding how Workspaces, Bindings, and Permissions interact at each lay
Join discussionFeb 11 · 4 min read · Metadata Is the Real Index: How Lakehouse Systems Avoid Scanning the World Today I want to talk about something that separates toy systems from production-grade data infrastructure: Metadata. Specifically, how modern lakehouse systems use metadata to...
Join discussion
Feb 8 · 3 min read · Modern data platforms promise scale, speed, and flexibility — yet they often feel overly complex.Data lakes, warehouses, pipelines, BI layers, governance tools — instead of simplifying analytics, many platforms seem to add more moving parts. This com...
Join discussion
Jan 17 · 9 min read · The data world, my friends, is in a fascinating state of flux. For years, we've chased the elusive "single source of truth," battling data silos and wrestling with the inherent trade-offs between the flexibility of data lakes and the robust governanc...
Join discussion
Jan 9 · 8 min read · Si trabajas con Apache Spark en Microsoft Fabric, probablemente te hayas enfrentado a la complejidad de optimizar configuraciones, reducir costos y mejorar el rendimiento de tus workloads. Sparkwise es una librería de Python diseñada específicamente ...
Join discussionDec 17, 2025 · 3 min read · 📜 What Is “Storage Wars”? As data volumes grow, organisations face a fundamental question:Where should data live once it’s ingested and prepared? Over time, three major storage architectures have emerged: Data Lakes for raw, large-scale data storag...
Join discussion
Dec 15, 2025 · 6 min read · En el universo de los datos, la presencia de duplicados es casi una garantía. Desde registros de clientes que se repiten hasta transacciones que aparecen más de una vez, los datos duplicados son un problema silencioso que puede socavar la fiabilidad ...
Join discussion
Dec 10, 2025 · 6 min read · Introduction Greetings! In the previous part, we defined the ETL architecture and analyzed the trade-offs without compromising the stated constraints. You can find Part 1 here: Data Ingestion Pipeline: A Data Engineer’s Dilemma and Azure Solutions. H...
Join discussion
