Watermarking and Late Data Handling in Spark Structured Streaming
Apr 19 · 27 min read · TLDR: A watermark tells Spark Structured Streaming: "I will accept events up to N minutes late, and then I am done waiting." Spark tracks the maximum event time seen per partition, takes the global mi
Join discussion





















