
It all started from a change in the checkpoint path of our Spark applications. We use Spark Structured streaming and AWS S3 buckets to maintain checkpoints. Let’s say we were using s3://bucket/spark/t
Sometimes you gotta run before you can walk
🧑🍳 Cooking data as a chef de partie at DataChef.co by day! 🪄 Avid coder inventing magic by night!
Markdown supported · ⌘↵ to post
No responses yet.