Dec 13, 2025 · 17 min read · Introduction What began as a weekend experiment — a simple curiosity born while testing the limits of my own laptop — has transformed into something much bigger. In recent months, pardoX has ceased to be merely a functional prototype and has evolved ...
Join discussionDec 10, 2025 · 25 min read · In this article, I’ll give you a beginner-friendly introduction to the Polars library in Python. Polars is an open-source library, originally written in Rust, which makes data wrangling easier in Python. The syntax of Polars is very similar to Pandas...
Join discussion
Jul 18, 2025 · 2 min read · There was question a couple of days ago on r/MicrosoftFabric subreddit on reading Data Warehouse tables shortcutted into Lakehouse. You can easily query this using Spark or T-SQL in the notebook, the question was how to do this using Polars since del...
Join discussionJun 8, 2025 · 7 min read · Streaming data processing has come a long way, so why stick to old methods and not use modern practices. Let me share my fresh perspective that can help you solve your problem. Inspiration from Batch Processing Batch Processing shines with below (tho...
Join discussion
May 28, 2025 · 2 min read · Every data team knows the drill: a PM needs to “just take a quick look” at some Parquet data. That usually means asking an engineer to write SQL or spin up a tool to pull a few rows. It’s a small ask, but one that happens often enough to slow everyon...
Join discussion
Apr 26, 2025 · 8 min read · Ever been bogged down by data pipelines crashing due to memory issues? It's a frustratingly common problem in data engineering projects. This post chronicles my experience of identifying and resolving memory bottlenecks in our data processing using t...
SMSKTFabio and 22 more commented
Dec 7, 2024 · 11 min read · In my last blog, I showed how to use Polars in a Fabric Python notebook to read, write and transform data. What I did not cover was how to write a Delta Lake table in a Python notebook that’s optimal for DirectLake performance. All engines in Fabric ...
KKay commentedDec 6, 2024 · 6 min read · 1. Convert Column Names to Snake Case (pyjanitor.clean_names) import pandas as pd import janitor # Sample DataFrame df = pd.DataFrame({"Column Name 1": [1, 2], "AnotherColumn": [3, 4]}) # Convert column names to snake_case df = janitor.clean_names(...
Join discussionDec 4, 2024 · 5 min read · The much-anticipated Python notebook in Fabric is finally available and the Fabric users have already developed cool libraries and blogged about the usefulness of these notebooks. Duckdb is everyone’s favorite, but I am a Python guy 😁 so here is qui...
MMichal commentedOct 7, 2024 · 16 min read · Apache Iceberg Crash Course: What is a Data Lakehouse and a Table Format? Free Copy of Apache Iceberg the Definitive Guide Free Apache Iceberg Crash Course Iceberg Lakehouse Engineering Video Playlist Data engineers and scientists often work wit...
Join discussion