The importance of not using nulls for merge keys

Last week I was working on a Spark pipeline that was running slowly, and I discovered that a specific task with significant skew was the cause. Googling for the problem didn’t return any meaningful result, so I had to figure it out myself. Here’s wha...