Sparks Fly
File Formats In the realm of data storage and processing, file formats play a pivotal role in defining how information is organized, stored, and accessed. These formats, ranging from simple text files to complex structured formats, serve as the blue...
#spark#apache-spark#parquet#avro#orc#sparksql#json#csv#serialization#deserialization#encoding#python#pyspark#file-formats
Responses
Markdown supported · ⌘↵ to post
Kolisetty Sasiram
BigData Enthusiast
Its great blog. Small doubt, In disk how spark stores the data is it in serialized form (byte format) or de-serialized form (object format). I felt this one is not clearly mentioned in the blog ? Because data spilling involves both serialization and de-serialization