Module 6 of Data Engineering Zoomcamp done!
Batch processing with Spark ๐ฅ
PySpark & DataFrames
Parquet file optimization
Spark UI on port 4040
Free course by @DataTalksClub: https://github.com/DataTalksClub/data-engineering-zoomcamp/
sykrebor.hashnode.dev1 min read