@kdarbha

The Art of Program Visibility: Managing Databricks + Azure Data Programs at Scale

6d ago · 11 min read · The Invisible Failure Most data platform programs don't fail loudly. They fail quietly — one missed dependency, one unreported pipeline issue, one status update that said 'green' while the underlying

Letting Go of Control: What Advaita Teaches Us About Liquid Clustering

May 5 · 11 min read · There is a pattern in mature engineering: the more you try to control, the more brittle the system becomes. Liquid Clustering is Delta Lake's answer to over-engineering. Vedanta figured this out mille

Unity Catalog - the Unified Self

Apr 28 · 15 min read · By Karthik Darbha | Tech4Nirvana The Problem of Many Selves In the Advaita Vedanta tradition, the root cause of all suffering is Avidya (अविद्या) — ignorance. Not ignorance in the ordinary sense of n

Migrating SPC Run Rules from SAS to Databricks

Apr 22 · 18 min read · A Pharma Supply Chain Engineering Perspective · tech4nirvana.com Why This Migration Is Non-Trivial Earlier, I worked as Product Owner and Data Architect on a SAS to Databricks migration for a Pharma

The Data Engineer's Vedanta: Ancient Wisdom for Modern Data Pipelines

Apr 19 · 7 min read · Introduction There is an ancient Sanskrit phrase that has guided seekers of truth for over a thousand years: "Tat Tvam Asi" — You are That. At its core, Advaita Vedanta, the non-dualist school of Indi