VBVishal Barvaliyainvishalbarvaliya.hashnode.dev·Sep 18, 2024 · 6 min readWhy Does the "Executor Out of Memory" Error Happen in Apache Spark?Apache Spark is a tool used to process large amounts of data. It’s fast, scalable, and great for big data tasks. However, sometimes when working with Spark, you might run into a common issue: the "Executor Out of Memory" error. If you've seen this er...00
VBVishal Barvaliyainvishalbarvaliya.hashnode.dev·Sep 18, 2024 · 5 min readCoding vs Programming: What’s the Difference?There’s a common myth in tech that coding and programming are the same thing, but they’re not. Many people use these words interchangeably, which can cause confusion. If you’re starting in tech or even if you’ve been around for a while, understanding...00
VBVishal Barvaliyainvishalbarvaliya.hashnode.dev·Sep 17, 2024 · 5 min readHow to Remove Leading Zeros from a Column in SQLWhen working with SQL databases, you might encounter numbers that contain leading zeros, such as 000123 or 0000456. While these zeros don't change the actual value of the number, they can make your data look unclean or cause issues when you're using ...00
VBVishal Barvaliyainvishalbarvaliya.hashnode.dev·Sep 17, 2024 · 2 min readKPMG Pyspark interview questions for Data Engineer 2024.Image Source How do you deploy PySpark applications in a production environment? What are some best practices for monitoring and logging PySpark jobs? How do you manage resources and scheduling in a PySpark application? Write a PySpark job to per...00
VBVishal Barvaliyainvishalbarvaliya.hashnode.dev·Sep 17, 2024 · 5 min readWhy Do MAANG Companies Ask Data Structures and Algorithms in Interviews for Data Engineers?If you’ve been preparing for a data engineering job at companies like Meta, Amazon, Apple, Netflix, or Google—often called MAANG—you might have noticed something surprising. They don’t just ask questions about tools like SQL, Python, or cloud platfor...00