---
title: "#spark articles | Hashnode"
description: "Explore 321 articles tagged #spark on Hashnode. Tutorials, guides, and how-tos from the community."
url: "https://hashnode.com/tag/spark"
---

Tag

# #spark

321 posts · 268 followers

[Dragos Roua](https://hashnode.com/@dragosroua) · [Oct 29](https://hashnode.com/posts/video-testing-spark-x2-6-4b-on-my-16gb-macbookpro/6ae30616bbb7f8db51bb1e7c)

### [\[ Video \] Testing Spark-X2.6–4B on my 16GB MacBookPro](https://dragosroua.hashnode.dev/video-testing-spark-x2-6-4b-on-my-16gb-macbookpro?utm_source=hashnode&utm_medium=feed)

Spark 2.6 4B from Spark LLM is a decent model for its size. In my testing, there was even room left for other apps on my MacBook Pro

2 min read · [dragosroua.hashnode.dev](https://dragosroua.hashnode.dev/video-testing-spark-x2-6-4b-on-my-16gb-macbookpro?utm_source=hashnode&utm_medium=feed)

[UpStep](https://hashnode.com/@upstep) · [Oct 10](https://hashnode.com/posts/does-this-etl-job-really-need-spark/6ac9d51fdfa98dbccdee760d)

### [Does This ETL Job Really Need Spark?](https://upstepasia.hashnode.dev/does-this-etl-job-really-need-spark?utm_source=hashnode&utm_medium=feed)

Compare DuckDB and Spark for ETL. Learn workload trade-offs, AWS Glue considerations, and a practical project to test performance and correctness.

8 min read · [upstepasia.hashnode.dev](https://upstepasia.hashnode.dev/does-this-etl-job-really-need-spark?utm_source=hashnode&utm_medium=feed)

[Satyam Prajapati](https://hashnode.com/@satyamprajapati) · [Sep 30](https://hashnode.com/posts/data-engineering-tools-astras-what-they-do-where-they-fit/6abe3c6c5f07edb1220035a7)

### [Data Engineering Tools(Astras) — What They Do & Where They Fit](https://datawithsatyam.hashnode.dev/data-engineering-tools-astras-what-they-do-where-they-fit?utm_source=hashnode&utm_medium=feed)

Explore the tools used in Data Engineering, what they do, where they fit in a data pipeline, and how they work together.

7 min read · [datawithsatyam.hashnode.dev](https://datawithsatyam.hashnode.dev/data-engineering-tools-astras-what-they-do-where-they-fit?utm_source=hashnode&utm_medium=feed)

[Pranjal Verma](https://hashnode.com/@pvcodes) · [Aug 23](https://hashnode.com/posts/apache-iceberg-the-backbone-of-the-modern-open-lakehouse/6a8a6ae172d37ec77f8066ae)

### [Apache Iceberg: The Backbone of the Modern Open Lakehouse](https://blog.pvcodes.in/apache-iceberg-the-backbone-of-the-modern-open-lakehouse?utm_source=hashnode&utm_medium=feed)

How snapshots, REST catalogs, and a disciplined maintenance pipeline turn raw object storage into an operable multi-engine lakehouse

6 min read · [blog.pvcodes.in](https://blog.pvcodes.in/apache-iceberg-the-backbone-of-the-modern-open-lakehouse?utm_source=hashnode&utm_medium=feed)

[Manu Shukla](https://hashnode.com/@mr-manushukla) · [Aug 22](https://hashnode.com/posts/clean-rooms-log-export-11-august-2026-every-count-is-rounded-down-and-stack-traces-are-gone/6a89a9c660a30bed105cd5bc)

### [Clean Rooms log export, 11 August 2026: every count is rounded down and stack traces are gone](https://ecorpit.hashnode.dev/clean-rooms-log-export-11-august-2026-every-count-is-rounded-down-and-stack-traces-are-gone?utm_source=hashnode&utm_medium=feed)

Clean Rooms log export, 11 August 2026: every count is rounded down and stack traces are gone Summary. On 11 August 2026 AWS Clean Rooms added export of Spark analysis logs for SQL queries, pitched as

15 min read · [ecorpit.hashnode.dev](https://ecorpit.hashnode.dev/clean-rooms-log-export-11-august-2026-every-count-is-rounded-down-and-stack-traces-are-gone?utm_source=hashnode&utm_medium=feed)

[Pranjal Verma](https://hashnode.com/@pvcodes) · [Aug 12](https://hashnode.com/posts/spark-performance-tuning-interview-prep-guide/6a7c3a02c4b28467692fba66)

### [Spark Performance Tuning Interview Prep Guide](https://blog.pvcodes.in/spark-performance-tuning-interview-prep-guide?utm_source=hashnode&utm_medium=feed)

Master Spark tuning for interviews: shuffle, partitioning, caching, joins, AQE, memory, Spark UI, and production scenarios for data engineering roles.

13 min read · [blog.pvcodes.in](https://blog.pvcodes.in/spark-performance-tuning-interview-prep-guide?utm_source=hashnode&utm_medium=feed)

[Dor Rafael Avrahamof](https://hashnode.com/@dorrafael) · [Aug 8](https://hashnode.com/posts/why-i-stopped-guessing-at-spark-and-dbt-config-values/6a779bd039637db4e99f36ad)

### [Why I stopped guessing at Spark and dbt config values](https://optipipe.hashnode.dev/why-i-stopped-guessing-at-spark-and-dbt-config-values?utm_source=hashnode&utm_medium=feed)

I've spent more than a decade building data pipelines, and the part nobody warns you about isn't the pipeline logic. It's the tuning. Executor memory, shuffle partitions, cluster size, thread counts.

4 min read · [optipipe.hashnode.dev](https://optipipe.hashnode.dev/why-i-stopped-guessing-at-spark-and-dbt-config-values?utm_source=hashnode&utm_medium=feed)

[areebakhizer7](https://hashnode.com/@areebakhizer) · [Aug 3](https://hashnode.com/posts/home-assistant-data-platform-prelude/6a7085b6b26ad397edfca448)

### [Unlocking Home Assistant Data: Building a Query Platform Part 1 — The Prelude](https://areebakhizer.hashnode.dev/home-assistant-data-platform-prelude?utm_source=hashnode&utm_medium=feed)

HA deletes all history after 10 days. I built a pipeline to capture, store, and query it permanently.

4 min read · [areebakhizer.hashnode.dev](https://areebakhizer.hashnode.dev/home-assistant-data-platform-prelude?utm_source=hashnode&utm_medium=feed)

[Rithwik Kumar Nagulapati](https://hashnode.com/@rithwikn) · [Jul 5](https://hashnode.com/posts/how-i-cut-infrastructure-costs-by-building-an-auto-shutdown-mechanism-for-a-long-running-java-batch-pipeline-on-an-ephemeral-cluster/6a4ae7e61c97d37e56e15f70)

### [How I Cut Infrastructure Costs by Building an Auto-Shutdown Mechanism for a Long-Running Java Batch Pipeline on an Ephemeral Cluster](https://rithwikn.hashnode.dev/how-i-cut-infrastructure-costs-by-building-an-auto-shutdown-mechanism-for-a-long-running-java-batch-pipeline-on-an-ephemeral-cluster?utm_source=hashnode&utm_medium=feed)

A production story about cost reduction, Quartz scheduler design, and gracefully shutting down a Java application that was never built to stop itself.

7 min read · [rithwikn.hashnode.dev](https://rithwikn.hashnode.dev/how-i-cut-infrastructure-costs-by-building-an-auto-shutdown-mechanism-for-a-long-running-java-batch-pipeline-on-an-ephemeral-cluster?utm_source=hashnode&utm_medium=feed)

[yadavlakshyageet](https://hashnode.com/@-lakshya) · [Jun 18](https://hashnode.com/posts/day-14-a-chill-day-assignment-progress-and-back-to-building/6a3401402d552f9171a3963e)

### [Day 14 — A Chill Day, Assignment Progress, and Back to Building](https://lakshyajourney.hashnode.dev/day-14-a-chill-day-assignment-progress-and-back-to-building?utm_source=hashnode&utm_medium=feed)

Today was one of those rare chill Thursdays. No classes. No work. Just had a mate come over, relaxed a bit, and enjoyed having a slower day for once. Not every day has to be a grind-fest. That said, I

2 min read · [lakshyajourney.hashnode.dev](https://lakshyajourney.hashnode.dev/day-14-a-chill-day-assignment-progress-and-back-to-building?utm_source=hashnode&utm_medium=feed)
