Tables in a Microsoft Fabric lakehouse are based on the Delta Lake table format. Delta Lake is an open-source storage layer for Spark that enables relational database capabilities for batch and streaming data. By using Delta Lake, you can implement a...
Markdown supported · ⌘↵ to post
Seems like DFg2 is using single thread to write. That explains single file and the time taken. Also, I expected ~600 Mil rows, i.e. all rows in an Row Group (RG) where there is only a single RG. Am I missing something?
Lamin Jobe
"Moving the needle"
Excellent article!