---
title: "Ayush Kumar (@ak2312) | Hashnode"
description: "I'm a Senior Data Engineer at Mu Sigma, currently working on large-scale supply chain data pipelines for a global FMCG client. My day-to-day involves buildin..."
url: "https://hashnode.com/@ak2312"
---

@ak2312

# Ayush Kumar

@ak2312 · Bengaluru, India · Joined August 2026

### [Building Incremental Pipelines with Auto Loader and Delta Lake on Azure Databricks](https://dataengineeringwithayush.hashnode.dev/building-incremental-pipelines-with-auto-loader-and-delta-lake-on-azure-databricks?utm_source=hashnode&utm_medium=feed)

A practical step-by-step guide to ingesting files incrementally without managing state yourself

7 min read · [dataengineeringwithayush.hashnode.dev](https://dataengineeringwithayush.hashnode.dev/building-incremental-pipelines-with-auto-loader-and-delta-lake-on-azure-databricks?utm_source=hashnode&utm_medium=feed)

[Aug 10](https://hashnode.com/posts/building-incremental-pipelines-with-auto-loader-and-delta-lake-on-azure-databricks/6a799d3b450d127a5189ce05)

### [Why Most Data Quality Frameworks Fail in Production (And What to Do Instead)](https://dataengineeringwithayush.hashnode.dev/why-most-data-quality-frameworks-fail-in-production-and-what-to-do-instead?utm_source=hashnode&utm_medium=feed)

Building a DQ framework is the easy part. Making it survive contact with real pipelines is where most teams struggle.

7 min read · [dataengineeringwithayush.hashnode.dev](https://dataengineeringwithayush.hashnode.dev/why-most-data-quality-frameworks-fail-in-production-and-what-to-do-instead?utm_source=hashnode&utm_medium=feed)

[Aug 10](https://hashnode.com/posts/why-most-data-quality-frameworks-fail-in-production-and-what-to-do-instead/6a799c20bab37548f7d2cf0f)

### [Unity Catalog on Azure Databricks: A Practical Setup Guide for Data Engineers](https://dataengineeringwithayush.hashnode.dev/unity-catalog-on-azure-databricks-a-practical-setup-guide-for-data-engineers?utm_source=hashnode&utm_medium=feed)

Beyond the official docs — what actually trips you up when setting up Unity Catalog in a real Azure environment

6 min read · [dataengineeringwithayush.hashnode.dev](https://dataengineeringwithayush.hashnode.dev/unity-catalog-on-azure-databricks-a-practical-setup-guide-for-data-engineers?utm_source=hashnode&utm_medium=feed)

[Aug 10](https://hashnode.com/posts/unity-catalog-on-azure-databricks-a-practical-setup-guide-for-data-engineers/6a799aee298925fe72506ac7)

### [Delta Lake Change Data Feed Is Underrated — Here's When to Actually Use It](https://dataengineeringwithayush.hashnode.dev/delta-lake-change-data-feed-is-underrated-here-s-when-to-actually-use-it?utm_source=hashnode&utm_medium=feed)

Most tutorials show you how to enable it. Few tell you when it's actually worth it.

5 min read · [dataengineeringwithayush.hashnode.dev](https://dataengineeringwithayush.hashnode.dev/delta-lake-change-data-feed-is-underrated-here-s-when-to-actually-use-it?utm_source=hashnode&utm_medium=feed)

[Aug 10](https://hashnode.com/posts/delta-lake-change-data-feed-is-underrated-here-s-when-to-actually-use-it/6a7999c86840574ac27396e8)

### [Selecting the Right Databricks Runtime Version for Production Pipelines](https://dataengineeringwithayush.hashnode.dev/selecting-the-right-databricks-runtime-version-for-production-pipelines?utm_source=hashnode&utm_medium=feed)

A practical guide for data engineers who don't want surprises in prod

4 min read · [dataengineeringwithayush.hashnode.dev](https://dataengineeringwithayush.hashnode.dev/selecting-the-right-databricks-runtime-version-for-production-pipelines?utm_source=hashnode&utm_medium=feed)

[Aug 10](https://hashnode.com/posts/selecting-the-right-databricks-runtime-version-for-production-pipelines/6a7999392763903fbfa7f592)
