Sandeep Pawarfabric.guru·Dec 7, 2024Delta Lake Tables For Optimal Direct Lake Performance In Fabric Python NotebookIn my last blog, I showed how to use Polars in a Fabric Python notebook to read, write and transform data. What I did not cover was how to write a Delta Lake table in a Python notebook that’s optimal for DirectLake performance. All engines in Fabric ...2 likes·1.6K readsmicrosoftfabric
Anix Lynchgozeroshot.dev·Dec 6, 2024Python Automation #2: 🗳️ Data Transformation w/polars, pyjanitor, pandas, polars1. Convert Column Names to Snake Case (pyjanitor.clean_names) import pandas as pd import janitor # Sample DataFrame df = pd.DataFrame({"Column Name 1": [1, 2], "AnotherColumn": [3, 4]}) # Convert column names to snake_case df = janitor.clean_names(...pyjanitor
Sandeep Pawarfabric.guru·Dec 4, 2024Working With Delta Tables In Fabric Python Notebook Using PolarsThe much-anticipated Python notebook in Fabric is finally available and the Fabric users have already developed cool libraries and blogged about the usefulness of these notebooks. Duckdb is everyone’s favorite, but I am a Python guy 😁 so here is qui...1.2K readspython notebook
Alex Mercedalexmerced.hashnode.dev·Oct 7, 2024Exploring Data Operations with PySpark, Pandas, DuckDB, Polars, and DataFusion in a Python NotebookApache Iceberg Crash Course: What is a Data Lakehouse and a Table Format? Free Copy of Apache Iceberg the Definitive Guide Free Apache Iceberg Crash Course Iceberg Lakehouse Engineering Video Playlist Data engineers and scientists often work wit...Python
Filippo Mamelimameli.hashnode.dev·Oct 6, 2024FeaturedRiding the Rust Wave: A Journey from Pandas to Polars in Data AnalysisThere is an interesting trend in the data community involving the rebuilding of popular products using Rust. Polars is one example of this, and it has really caught my interest. YADL (Yet Another Dataframe Library)? I've seen many articles claiming t...21 likes·219 readsRust
Anix Lynchgozeroshot.dev·Oct 3, 202420 Polars concepts with Before-and-After Examples1. Creating DataFrames (pl.DataFrame) 🏗️ Boilerplate Code: import polars as pl Use Case: Create a DataFrame to hold your data, similar to pandas. 🏗️ Goal: Store and manipulate data in a high-performance DataFrame structure. 🎯 Sample Code: # Creat...Matplotlib
Jason Shiersrandomforest.hashnode.dev·Sep 7, 2024NS&I Premium Bonds: Insights from a Monte Carlo ExperimentIn Part 1 of this series, we explored the characteristics of NS&I Premium Bonds and created a function to simulate the results of a single monthly prize draw for a given bond holding. In Part 2, we built a Monte Carlo experiment, simulated 6 million ...Monte Carlo Simulation of NS&I Premium Bondsmonte carlo
Sandeep Pawarfabric.guru·Aug 4, 2024Quick Test : Daft With Ray In Fabric💡 This is not a benchmark. This is one specific instance of a test, on a specific dataset using specific transformations. My goal was just to see how to set up daft + ray in Fabric and compare for the sake of learning. Daft is a distributed query ...457 readsquicktest
Noufal Salimblog.noufals.in·Jun 9, 2024Unlocking the Potential: Evaluating the Best Data Processing Frameworks for Your Needs - A Comparative Study of Pandas, Dask, and PolarsAbstract As the field of data processing and analysis continues to advance, it is becoming increasingly crucial to select the appropriate tools for the task at hand. The purpose of this white paper is to present an extensive comparison of three well-...Polars
ayoub arahmatayoubar.hashnode.dev·Jan 7, 2024How do I convert a Django QuerySet into a Polars Dataframe?The Problem: In my software engineering journey, I faced a problem while working on a Django project when I had to grab a bunch of data from a PostgreSQL server and mess around with it to do some data analysis. The usual way to do this is by running ...118 readsDjango
