---
title: "Discussion on \"Apache Spark - Tutorial 3\" | Hashnode"
description: "Discussion on \"Apache Spark - Tutorial 3\". Here we are going to learn Spark Memory Management\nBefore starting we need to understand the below points clearly,\n\nOne core will process one partition of data at a time\n\nSpark partition is equivalent to HDFS blocks and repartition is possible\n\nOne t..."
url: "https://hashnode.com/posts/apache-spark-tutorial-3/63a877a3516640481d6bb5cc"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Apache Spark - Tutorial 3](https://sivayuvi79.hashnode.dev/apache-spark-tutorial-3?utm_source=hashnode&utm_medium=feed)

Here we are going to learn Spark Memory Management Before starting we need to understand the below points clearly, One core will process one partition of data at a time Spark partition is equivalent to HDFS blocks and repartition is possible One t...

Dec 25, 2022 · 4 min read · sivayuvi79.hashnode.dev  
Read the article

-   [#spark](https://hashnode.com/tag/spark)
-   [#spark-for-data-science](https://hashnode.com/tag/spark-for-data-science)
-   [#memory-management](https://hashnode.com/tag/memory-management)
-   [#pyspark](https://hashnode.com/tag/pyspark)

## Written by

[Sivaraman Arumugam](https://hashnode.com/@SIVARAMAN79)

@SIVARAMAN79 · Chennai, India

  
[View profile](https://hashnode.com/@SIVARAMAN79)

Writes about Bigdata, System Design, DSA, Python and SQL

I am a data engineer who is responsible for designing, building, maintaining, and testing the infrastructure and systems that are used to store, process, and analyze data. I work closely with data scientists and analysts to ensure that the data pipelines and systems are able to support the data needs of an organization. I have a strong background in computer science and software engineering, and skilled in programming languages such as Python, Java, and SQL also familiar with database systems and big data technologies like Hadoop, Spark, and NoSQL databases. Some of my key responsibilities as a data engineer: Designing and building data pipelines to extract, transform, and load data from various sources Setting up and maintaining data storage and processing systems, including data warehouses and data lakes Collaborating with data scientists and analysts to understand their data needs and ensure that the data infrastructure can support their requirements Performing data quality checks and troubleshooting any issues that arise Implementing security and privacy measures to protect sensitive data

## Responses

Markdown supported · ⌘↵ to post

No responses yet.
