AAnirudh Singh Bhadauria·Jul 21Stop Building ETL Pipelines. Start Declaring Them ✨Discover how Spark Declarative Pipelines SDP replaces complex ETL orchestration with Auto Loader, Auto CDC & Append Flows for simple data engineering6 min read·anirudhbhadauria.hashnode.dev
VVishal Uttam Mane·Apr 17Real-Time Data Analytics with Streaming Platforms This article explains how real-time data analytics leverages streaming platforms to process and analyze data instantly using distributed,4 min read·visha-uttam-mane-real-time-data.hashnode.dev
PPriyanshu Sharma·Jan 11Kafka Streams vs Flink vs Spark Streaming — A Senior System Design Interview PerspectiveKafka Streams vs Flink vs Spark Streaming — A Senior System Design Interview Perspective2 min read·dataplatform.hashnode.dev
AAkash Jain·May 21, 2025Demystifying Lambda and Kappa architecturesExplore Lambda architecture vs. Kappa architecture in this deep dive. Understand their layers, benefits, and challenges for big data processing, from histor7 min read·blog.akashja.in
AAjay Veerabomma·Jun 24, 2024Comparing Python Consumers and Spark Structured Streaming: Reading from Kinesis and KafkaPython consumers and Spark Structured Streaming differ in mechanisms, capabilities, and use cases with Kinesis and Kafka6 min read·ajayveerabomma.com
JJanga Venkata Phanindra Reddy·Jun 5, 2023Streaming Data Pipeline Using Confluent Cloud and Dataproc on GCPIntroduction In This Article, we'll look into How we can build a streaming data pipeline using Confluent Kafka and Spark Structured Streaming on Dataproc(GCP). Objective: Create a Confluent Kafka Topic and Stream events from Kafka to Google cloud sto...3 min read·phanindrareddyjv.hashnode.dev
WWarui Wanjiru·Apr 30, 2023Setting up a Spark Cluster to Read Messages from Kafka(Beat~lytica part 2)Welcome to my second installation of the Beatlytica Series. Welcome back, fellow data enthusiasts! I'm thrilled to share with you the second installation of the Beatlytica blog series, where I dive into the exciting world of real-time data streaming ...3 min read·waruithemystery.hashnode.dev