Data Engineer | ETL, Data Quality, Azure, Python, SQL, Spark | Building Scalable & Reliable Data Systems
Open to work on data engineering world, exploring roles.
Sep 22, 2025 · 4 min read · Data Flow is essentially a Spark V2 version of clusters that helps us perform data transformation operations in cloud services like Azure. Before Data Flow, if you wanted to perform transformations, you could simply use joins and other operations on ...
Join discussionSep 16, 2025 · 4 min read · Azure is a cloud-based platform that offers many services like IaaS (Infrastructure as a Service), PaaS (Platform as a Service), and SaaS (Software as a Service). ADF ( Azure Data Factory ) is a Paas i.e, Platform as a service which leverages our clo...
Join discussionSep 14, 2025 · 12 min read · Airflow UI provides a comprehensive view of pipeline execution, including task status, logs, alerts, and information on failed or interrupted pipelines. Automating Data Pipelines :- Four core data orcherstration we have been discussed earlier like Wo...
Join discussion