6d ago · 5 min read · Building a Better Rules Data Platform on Snowflake for Government The New Zealand Government's Better Rules for Government discovery report outlined a compelling vision: transform legislation, regulations, and policy into machine-consumable formats t...
Join discussion
Apr 8 · 18 min read · { "@context": "https://schema.org", "@type": "FAQPage", "mainEntity": [ { "@type": "Question", "name": "What is the real-time data stack?", "acceptedAnswer": { "@type": "Answer", "text": "The real-time data...
Join discussion
Apr 3 · 7 min read · A medallion architecture with RisingWave and Iceberg uses three tiers: Bronze (raw data landed in Iceberg via RisingWave sinks), Silver (cleaned and joined data in Iceberg, computed by RisingWave materialized views), and Gold (business-ready aggregat...
Join discussion
Apr 3 · 6 min read · A streaming lakehouse combines real-time stream processing with open-format data lake storage. RisingWave handles the streaming SQL layer — ingesting from Kafka and databases, applying continuous transformations — while Apache Iceberg provides ACID-c...
Join discussion
Mar 25 · 10 min read · The Challenge: Zero to Production in 72 Hours In January 2025, we faced an ambitious challenge: build a production-ready data platform capable of ingesting, processing, and storing diagnostic data from multiple sources at massive scale. The requireme...
Join discussionMar 12 · 5 min read · Introduction Large Language Models (LLMs) have rapidly become the centerpiece of many AI systems. From chatbots to code assistants, they enable natural language interaction and powerful generative cap
Join discussion
Feb 19 · 9 min read · Medallion architecture has become the go-to standard for organizing data in a lakehouse — yet many teams overlook its full potential. It's common to find Silver tables that are straight copies of Bron
Join discussionFeb 12 · 3 min read · When working with large-scale data in Apache Spark, understanding join strategies is critical for performance tuning. Spark does not always execute joins the same way. Depending on dataset size and co
Join discussionFeb 8 · 3 min read · Modern data platforms promise scale, speed, and flexibility — yet they often feel overly complex.Data lakes, warehouses, pipelines, BI layers, governance tools — instead of simplifying analytics, many platforms seem to add more moving parts. This com...
Join discussion
Jan 25 · 37 min read · 📖 Table of Contents Introduction Row-Based Formats CSV JSON Avro Sequence File Column-Based Formats Parquet ORC Hybrid Formats Delta Lake Apache Hudi Apache Iceberg Specialized Formats Protocol Buffers (Protobuf) Thrift MessagePack Apache Arr...
Join discussion
