Thanks for the clarification 😊 Great blog and great work from your side. Pls continue it

I think, sort-merge join also, shuffling happens. In sort merge join, shuffle is the 1st step, then in each executor this sorting and merge operation will be performed I think. Pls correct me If I am wrong.

Its great blog. Small doubt, In disk how spark stores the data is it in serialized form (byte format) or de-serialized form (object format). I felt this one is not clearly mentioned in the blog ? Because data spilling involves both serialization and de-serialization

Good one.. May be along with DAGs .. lineage graph should also include , then Spark immutable , resilience etc..