3h ago · 10 min read · How to Build a Real-Time SIEM Data Pipeline A real-time SIEM data pipeline ingests security logs from Kafka, normalizes them into a unified schema using materialized views, and applies continuous SQL-based detection rules that fire alerts within seco...
Join discussion
Apr 17 · 8 min read · Mastering Snowflake Data Engineering: 7 Pro Tips for High-Performance Pipelines Snowflake has rapidly become one of the most adopted data warehouse platforms in the enterprise world, and for good reason. Its unique architecture separates storage from...
Join discussion
Apr 3 · 6 min read · To sink Kafka streams to Apache Iceberg with RisingWave: create a Kafka source, build a materialized view to transform or aggregate the data, then create an Iceberg sink pointing to your catalog and table. RisingWave handles continuous writes automat...
Join discussion
Apr 3 · 5 min read · RisingWave supports multiple independent Iceberg sinks running concurrently from a single deployment. Each sink reads from its own materialized view and writes to a separate Iceberg table with its own schema and partition spec. This fan-out pattern e...
Join discussion
Mar 6 · 9 min read · Stack: AWS Glue · PySpark · Step Functions · APIs · Power BI Welcome to my very first technical blog post, I'm Omer and I love many things in life, but for now, you will know only two of them: detail
Join discussion
Feb 19 · 5 min read · We’ve all been there: it’s 3 AM, and your data pipeline has stalled. A website you’ve been scraping for months decided to wrap their price tag in an extra <div> or rename a CSS class from product-price to item-price-v2. Your scraper, built on a house...
Join discussion
Feb 17 · 12 min read · When your hotel database thinks "Game Room, Deck & Yard: Chicago Home" is a hotel, you have a data quality problem. When it happens across 212 cities in 25 countries, this isn’t a travel problem; it’s
Kklement and 1 more commented
