---
title: "Discussion on \"Project Tungsten in Spark: The Performance Work Happening Under the Hood\" | Hashnode"
description: "Discussion on \"Project Tungsten in Spark: The Performance Work Happening Under the Hood\". Spark can distribute your workload across hundreds of machines. But if every row wastes CPU cycles and memory, congratulations — you've distributed the inefficiency.\n\nLet's talk about Project Tungsten.\n\nIf you've worked with Spark long enough, you've probably heard the usual performance advice:\n\nUse DataFrames instead of writing everything with RDDs.\nAvoid Python UDFs when built-in Spark SQL functions can do the job.\nWatch your shuffles.\nTune executor memory.\nCheck the Spark UI before randomly changing configurations.\n\nAll good advice.\nBut there's a deeper question hiding underneath it.\n\nWhy can Spark SQL execute the same kind of data processing more efficiently than a straightforward implementation that creates and manipulates millions of JVM objects?\n\nThat question takes us to Project Tungsten.\n\nAnd if you're preparing for Senior Data Engineer, Lead Data Engineer, or Staff Engineer interviews, this is one of those topics worth understanding properly. Not because an interviewer might ask you to define UnsafeRow — although they might — but because Tungsten connects several important concepts: JVM memory, garbage collection, CPU efficiency, code generation, cache locality, and physical execution plans.\n\nThese concepts explain why some Spark jobs fly while others turn your cluster into a very expensive space heater.\n\nLet's get into it."
url: "https://hashnode.com/posts/spark-project-tungsten-internals/6ad5d05e6621b4e4a7994012"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Project Tungsten in Spark: The Performance Work Happening Under the Hood](https://core-de-platform.hashnode.dev/spark-project-tungsten-internals?utm_source=hashnode&utm_medium=feed)

Spark can distribute your workload across hundreds of machines. But if every row wastes CPU cycles and memory, congratulations — you've distributed the inefficiency. Let's talk about Project Tungsten. If you've worked with Spark long enough, you've probably heard the usual performance advice: Use DataFrames instead of writing everything with RDDs. Avoid Python UDFs when built-in Spark SQL functions can do the job. Watch your shuffles. Tune executor memory. Check the Spark UI before randomly changing configurations. All good advice. But there's a deeper question hiding underneath it. Why can Spark SQL execute the same kind of data processing more efficiently than a straightforward implementation that creates and manipulates millions of JVM objects? That question takes us to Project Tungsten. And if you're preparing for Senior Data Engineer, Lead Data Engineer, or Staff Engineer interviews, this is one of those topics worth understanding properly. Not because an interviewer might ask you to define UnsafeRow — although they might — but because Tungsten connects several important concepts: JVM memory, garbage collection, CPU efficiency, code generation, cache locality, and physical execution plans. These concepts explain why some Spark jobs fly while others turn your cluster into a very expensive space heater. Let's get into it.

Sep 28 · 17 min read · core-de-platform.hashnode.dev  
Read the article

## Written by

[srkhan2809](https://hashnode.com/@shahida-khan)

@shahida-khan · Dubai, UAE

  
[View profile](https://hashnode.com/@shahida-khan)

Lead Data Infrastructure Architect | Databricks, PySpark & Cloud DW

I am a Lead Data Infrastructure Architect with 13+ years of experience engineering high-throughput data platforms for fintech and consumer apps. I specialize in the Databricks Lakehouse ecosystem, PySpark, BigQuery, and Snowflake. I write about modern data architecture, cloud cost optimization (FinOps), declarative ELT pipelines, and scaling systems to process 100K+ events per second. When I'm not untangling complex legacy databases, I'm building zero-loss data quality frameworks.

## Responses

Markdown supported · ⌘↵ to post

No responses yet.