Oct 19, 2024 · 15 min read · Here are some common interview questions and answers related to transformations in Spark: 1. What are narrow and wide transformations in Spark? Answer: Narrow transformations are transformations where each partition of the parent RDD is used to produ...
Join discussionOct 19, 2024 · 3 min read · Conceptual Questions What is parallelize in PySpark? parallelize is a method in PySpark used to convert a Python collection (like a list or a tuple) into an RDD (Resilient Distributed Dataset). This allows you to perform parallel processing on the ...
Join discussionAug 26, 2024 · 3 min read · Apache Spark's execution model is one of the reasons it stands out as a powerful tool for big data processing. At its core, Spark's execution model revolves around two main concepts: transformations and actions. To understand how Spark operates, it’s...
Join discussion
Mar 30, 2023 · 4 min read · Posted by Renjitha K in Renjitha K's Blog on Mar 25, 2023 2:27:13 PM As the amount of data generated by individuals and businesses continue to grow exponentially, the need for technologies like Apache Spark that can process and analyze large dataset...
Join discussionJan 18, 2023 · 4 min read · RDD(resilient distributed datasets) are the basic unit of storage in spark. you can think of an rdd as a collection distributed over multiple machines.Most of the time higher level structured APIs are used in spark applications which under the hood g...
Join discussion