---
title: "#structured-streaming articles | Hashnode"
description: "Explore 6 articles tagged #structured-streaming on Hashnode. Tutorials, guides, and how-tos from the community."
url: "https://hashnode.com/tag/structured-streaming"
---

Tag

# #structured-streaming

6 posts · 0 followers

[Abstract Algorithms](https://hashnode.com/@abstractalgorithms) · [Apr 19](https://hashnode.com/posts/spark-kafka-structured-streaming-pipeline/69e437a6b8882bee29d621f8)

### [Kafka and Spark Structured Streaming: Building a Production Pipeline](https://abstractalgorithms.hashnode.dev/spark-kafka-structured-streaming-pipeline?utm_source=hashnode&utm_medium=feed)

Kafka delivers events, Spark processes them — offset management, schema evolution, and exactly-once delivery determine production reliability

23 min read · [abstractalgorithms.hashnode.dev](https://abstractalgorithms.hashnode.dev/spark-kafka-structured-streaming-pipeline?utm_source=hashnode&utm_medium=feed)

[Abstract Algorithms](https://hashnode.com/@abstractalgorithms) · [Apr 19](https://hashnode.com/posts/spark-watermarking-late-data-handling/69e437f990c8f3c452e31ff0)

### [Watermarking and Late Data Handling in Spark Structured Streaming](https://abstractalgorithms.hashnode.dev/spark-watermarking-late-data-handling?utm_source=hashnode&utm_medium=feed)

Late events are inevitable in distributed systems — watermarks tell Spark exactly when it is safe to finalize a window without waiting forever

27 min read · [abstractalgorithms.hashnode.dev](https://abstractalgorithms.hashnode.dev/spark-watermarking-late-data-handling?utm_source=hashnode&utm_medium=feed)

[Abstract Algorithms](https://hashnode.com/@abstractalgorithms) · [Apr 19](https://hashnode.com/posts/spark-stateful-aggregations-mapgroupswithstate/69e437e26e738bcf4b61fc35)

### [Stateful Aggregations in Spark Structured Streaming: mapGroupsWithState](https://abstractalgorithms.hashnode.dev/spark-stateful-aggregations-mapgroupswithstate?utm_source=hashnode&utm_medium=feed)

mapGroupsWithState gives you a per-key mutable state machine in a streaming pipeline — it is the right tool when windows are not enough

28 min read · [abstractalgorithms.hashnode.dev](https://abstractalgorithms.hashnode.dev/spark-stateful-aggregations-mapgroupswithstate?utm_source=hashnode&utm_medium=feed)

[Abstract Algorithms](https://hashnode.com/@abstractalgorithms) · [Apr 19](https://hashnode.com/posts/spark-structured-streaming-micro-batch-continuous/69e437ed6e738bcf4b61fc37)

### [Spark Structured Streaming: Micro-Batch vs Continuous Processing](https://abstractalgorithms.hashnode.dev/spark-structured-streaming-micro-batch-continuous?utm_source=hashnode&utm_medium=feed)

Treats streams as unbounded tables — the execution model determines whether your pipeline hits milliseconds or minutes.

27 min read · [abstractalgorithms.hashnode.dev](https://abstractalgorithms.hashnode.dev/spark-structured-streaming-micro-batch-continuous?utm_source=hashnode&utm_medium=feed)

[Shahin](https://hashnode.com/@Shahinism) · [Dec 20, 2023](https://hashnode.com/posts/mysterious-spark-checkpoints-behaviour/659e90ea5946a6311eb37de4)

### [Mysterious Spark Checkpoints Behaviour](https://blog.datachef.co/mysterious-spark-checkpoints-behaviour?utm_source=hashnode&utm_medium=feed)

It all started from a change in the checkpoint path of our Spark applications. We use Spark Structured streaming and AWS S3 buckets to maintain checkpoints. Let’s say we were using s3://bucket/spark/t

3 min read · [blog.datachef.co](https://blog.datachef.co/mysterious-spark-checkpoints-behaviour?utm_source=hashnode&utm_medium=feed)

[Shahin](https://hashnode.com/@Shahinism) · [Oct 12, 2023](https://hashnode.com/posts/composing-functions-for-pysparks-structured-streaming/6527a78a83fe36987a5bc299)

### [Composing Functions for PySpark's Structured Streaming](https://shahin.blog/composing-functions-for-pysparks-structured-streaming?utm_source=hashnode&utm_medium=feed)

In PySpark's structured streaming, striving for modular, reusable code while ensuring adaptability for each unique use case is a common challenge. I recently had the task of leveraging the broad capabilities of the writeStream method without losing t...

2 min read · [shahin.blog](https://shahin.blog/composing-functions-for-pysparks-structured-streaming?utm_source=hashnode&utm_medium=feed)
