Postmortem of Spark Executor's death
A question I kept coming back to while comparing Spark with BigQuery was this:
If Spark executors write shuffle data to disk, and that disk still exists, why can’t other executors read that data when one executor dies?
At first glance, it feels lik...
bytesofdeepankar.hashnode.dev4 min read