Stateful Aggregations in Spark Structured Streaming: mapGroupsWithState

Apr 19 · 28 min read · TLDR: mapGroupsWithState gives each streaming key its own mutable state object, persisted in a fault-tolerant state store that checkpoints to object storage on every micro-batch. Where window aggregat