Mar 24 · 9 min read · At FabCon Atlanta last week, the updated notebook Copilot for data engineering and data science was announced. It brings agentic capabilities to the Copilot and is much more intelligent and Fabric-awa
Join discussionMar 16 · 20 min read · Where We Left Off Part 1 of this series stood up the Azure infrastructure and Databricks workspace via Terraform. Part 2 added the Unity Catalog layer — catalog, schemas, and grants — managed through
Join discussionMar 12 · 4 min read · Start with a relatable junior struggle: "Ever stared at a Spark UI wondering why your 'simple' PySpark job is shuffling 100GB for a 1GB dataset? Or why Delta reads take forever? I did – for years. As
Join discussion
Mar 6 · 9 min read · Stack: AWS Glue · PySpark · Step Functions · APIs · Power BI Welcome to my very first technical blog post, I'm Omer and I love many things in life, but for now, you will know only two of them: detail
Join discussion
Mar 5 · 4 min read · The problem-When Migration Breaks What Already Works For years, daily sales orders flowed from system A into our pipeline without issues. I was on the downstream team — we'd receive the data, run it t
Join discussionFeb 5 · 70 min read · In the world of big data, performance isn't just about bigger clusters – it's about smarter code. Spark is deceptively simple to write but notoriously difficult to optimize, because what you write isn't what Spark executes. Between your transformatio...
Join discussion
Jan 24 · 3 min read · Introduction I’m currently working as a Software Tester, and I’m starting this blog to document my journey into Data Engineering. This is not a blog written by an expert.It’s written by someone who is learning seriously, step by step, and wants to un...
Join discussionJan 16 · 4 min read · ¿Alguna vez has eliminado accidentalmente un workspace en Microsoft Fabric y has sentido ese momento de pánico? No te preocupes, Microsoft ha incorporado una API de administración que te permite restaurar workspaces eliminados. En este artículo te mo...
Join discussion
Jan 9 · 8 min read · Si trabajas con Apache Spark en Microsoft Fabric, probablemente te hayas enfrentado a la complejidad de optimizar configuraciones, reducir costos y mejorar el rendimiento de tus workloads. Sparkwise es una librería de Python diseñada específicamente ...
Join discussion
