Stateful Aggregations in Spark Structured Streaming: mapGroupsWithState

5d ago · 25 min read · TLDR: mapGroupsWithState gives each streaming key its own mutable state object, persisted in a fault-tolerant state store that checkpoints to object storage on every micro-batch. Where window aggregations assume fixed time boundaries, mapGroupsWithSt...