Muhammad Atif Hafeez

@matifhafeez

Software Architect

Joined December 2024

About

Nothing here yet.

Available for

Nothing here yet.

Muhammad Atif Hafeez's blogs

ADF: Connect to On-Prem Data Sourcesadf-connect-to-on-prem-data-sources.hashnode.dev1 post

Databricks Spark SQL - SCD Type 2databrickssparksqlscdtype2.hashnode.dev1 post

Customer Segmentationcustomersegmentation.hashnode.dev1 post

How a Library Broke My DB Clusterdatabricks-library-dependency-hell.hashnode.dev1 post

Articles Comments

Recently published

MAMuhammad Atif Hafeezdatabricks-library-dependency-hell.hashnode.devJul 10, 2025 · 3 min read

⚠️ How I Diagnosed and Fixed a Silent NumPy Upgrade That Broke My Databricks Pipelines

Background Recently, one of our critical ADF production pipelines failed unexpectedly with the following error from the Databricks notebook activity: "Could not reach driver of cluster 0619-120457-7gl1w8rb." The notebook had been working fine earli...

0

MAMuhammad Atif Hafeezcustomersegmentation.hashnode.devApr 23, 2025 · 3 min read

From Raw Data to Customer Insights: My Journey Building a Scalable Segmentation Model Using PySpark & K-Means

Intro: In this post, I’ll walk you through a customer segmentation project I recently completed using PySpark. The goal was to identify distinct customer groups based on behavioral and engagement features to support personalized marketing strategies...

0

MAMuhammad Atif Hafeezdatabrickssparksqlscdtype2.hashnode.devFeb 21, 2025 · 5 min read

Implementing SCD Type 2 Using Spark SQL and Delta Tables

In this blog, I will walk you through a straightforward approach to implementing Slowly Changing Dimension (SCD) Type 2 using Spark SQL and Delta Tables. This method ensures atomicity and maintains the history of records effectively. Introduction to ...

0

MAMuhammad Atif Hafeezadf-connect-to-on-prem-data-sources.hashnode.devDec 25, 2024 · 5 min read

Connect Azure Data Factory to On-Prem Data Sources

Introduction Azure Data Factory (ADF) is a powerful cloud-based data integration service that allows you to create, schedule, and orchestrate data workflows. In this blog, I’ll demonstrate how I extended an existing Azure Data Engineering project (or...

0

Muhammad Atif Hafeez

About

Available for

Muhammad Atif Hafeez's blogs

Recently published

⚠️ How I Diagnosed and Fixed a Silent NumPy Upgrade That Broke My Databricks Pipelines

From Raw Data to Customer Insights: My Journey Building a Scalable Segmentation Model Using PySpark & K-Means

Implementing SCD Type 2 Using Spark SQL and Delta Tables

Connect Azure Data Factory to On-Prem Data Sources

Search Hashnode

Muhammad Atif Hafeez

About

Available for

Muhammad Atif Hafeez's blogs

Recently published

⚠️ How I Diagnosed and Fixed a Silent NumPy Upgrade That Broke My Databricks Pipelines

From Raw Data to Customer Insights: My Journey Building a Scalable Segmentation Model Using PySpark & K-Means

Implementing SCD Type 2 Using Spark SQL and Delta Tables

Connect Azure Data Factory to On-Prem Data Sources