SSai Dinesh Konda路Jan 24From Software Testing to Data Engineering: My Learning JourneyDocumenting my transition into Data Engineering3 min read路learningdataengineering.hashnode.dev
DDeepankar Yadav路Dec 22, 2025Postmortem of Spark Executor's deathUnderstanding executor-local shuffle ownership in Apache Spark4 min read路bytesofdeepankar.hashnode.dev
AAbhishek Kumar路Nov 1, 2025Learning PySpark with DatabricksA Beginner-Friendly Guide to PySpark37 min read路abhi1213.hashnode.dev
JJidhun Puthuppattu路Aug 29, 2025Data Integration and Transformation Tools: A Comprehensive ExplorationExplore key data tools鈥擲park SQL, Airflow, Kafka鈥攆or scalable analytics and machine learning4 min read路jidhun.hashnode.dev
IIslam Elbanna路May 17, 2025Mastering Apache Spark SQL: Create Complex Queries with Common Table Expressions CTE (WITH Clause)Enhance Apache Spark SQL queries with CTEs for improved readability, performance, and reusability in large-scale data processing5 min read路practical-software.com
MMuhammad Atif Hafeez路Feb 21, 2025Implementing SCD Type 2 Using Spark SQL and Delta TablesIn this blog, I will walk you through a straightforward approach to implementing Slowly Changing Dimension (SCD) Type 2 using Spark SQL and Delta Tables. This method ensures atomicity and maintains the history of records effectively. Introduction to ...5 min read路databrickssparksqlscdtype2.hashnode.dev
SShreyash Bante路Aug 22, 2024Apache Spark ArchitectureWe are talking about spark architecture and how code is executed4 min read路bigdatageek.dev
KKumar Rohit路Aug 15, 2024Hello Spark on MinikubeSpark, Kubernetes, docker, k8s, minikube, kubernetes, k3s, docker-image, etl, spark-etl, data, data-engineering, data-platforms, containers, image13 min read路krohit-de.hashnode.dev
MMehul Kansal路Aug 12, 2024Week 12: Mastering Hive 馃搱Hey fellow data engineers! 馃憢 This week's blog explores the architecture of Hive, offering insights into its components like data storage and metadata management. It also covers the different types of tables Hive supports, essential optimizations for...7 min read路mehulkansal.hashnode.dev
TTurboline LTD路Jul 10, 2024AI-Based Data Transformation: A Comparison of LLM-Generated PySpark Code (Using Mistral & Google Gemini Advanced)Mistral Le Chat Mistral's Le Chat didn't have the functionality to upload CSV files, so I couldn't generate a transformation code. I could have built a lightweight ingestion process using Mistral API and Langchain. However, that was out of scope for ...4 min read路blog.turboline.ai