BDBiju Devassyinbijudevassy.hashnode.dev·Feb 22 · 5 min readCaching vs Persistence in Spark (PySpark)Introduction Apache Spark is built on lazy evaluation. Transformations such as select, filter, join, and groupBy do not execute immediately. Instead, Spark builds a logical plan (DAG) and executes it 00
BDBiju Devassyinbijudevassy.hashnode.dev·Feb 12 · 3 min readBroadcast Join vs Sort Merge Join vs Shuffle Hash Join in Apache SparkWhen working with large-scale data in Apache Spark, understanding join strategies is critical for performance tuning. Spark does not always execute joins the same way. Depending on dataset size and co00
