Peer Grønneruppeerinsights.hashnode.dev·Oct 28, 2024Automating Fabric: Kickstart your Fabric Data Platform setupSetting up and managing workspaces in Microsoft Fabric can be a time-consuming task, especially when you need multiple workspaces for various stages of the data lifecycle across different environments. This blog post demonstrates how to streamline yo...Discuss·363 readsmicrosoftfabric
Sandeep PawarProfabric.guru·Sep 19, 2024Programmatically Removing & Updating Default Lakehouse Of A Fabric NotebookI have written about default lakehouse of a Fabric notebook before here and here. However, unless you used the notebook API, there was no easy/quick way of removing all/selective lakehouses or updating the default lakehouse of a notebook. But thanks ...Discuss·1.3K readsnotebookutils
Sandeep PawarProfabric.guru·Sep 4, 2024Querying SQL Endpoint of Fabric Lakehouse/Warehouse In A Notebook With T-SQLI am not sharing anything new. The spark data warehouse connector has been available for a couple months now. It had some bugs, but it seems to be stable now. This connector allows you to query the lakehouse or warehouse endpoint in the Fabric notebo...Discuss·1.9K readssql endpoint
Sandeep PawarProfabric.guru·Aug 26, 2024Quick Tip : Using NotebookUtils To Get Fabric Lakehouse PropertiesNotebookutils in Fabric now has getWithProperties() method to get the properties of a lakehouse. Earlier the get() method only returned abfs path and not the SQL endpoint. By default it returns the JSON: //sample output {'id': '<>', 'type': 'Lake...Discuss·1.2K readsmicrosoftfabric
Sandeep PawarProfabric.guru·Jul 30, 2024Using fsspec To Define OneLake Filesystem in FabricI mentioned on X the other day that, like other filesystem backends such as S3 and GCS, you can use fsspec to define the OneLake filesystem too. In this blog, I will explain how to define it and why it's important to know about it. https://x.com/Pawa...Sachin Nandanwar and 1 other are discussing this2 people are discussing thisDiscuss·2 likes·685 readsdaft
Sandeep PawarProfabric.guru·Jul 24, 2024How to Attach a Default Lakehouse to a Notebook in FabricI wrote a blog post a while ago on mounting a lakehouse (or generally speaking a storage location) to all nodes in a Fabric spark notebook. This allows you to use the File API file path from the mounted lakehouse. Mounting a lakehouse using mssparkut...Frithjof Vassbø and 2 others are discussing this3 people are discussing thisDiscuss·1 like·2.3K readsmicrosoftfabric
Sandeep PawarProfabric.guru·Jul 19, 2024Calculating The Size of The Dataflow Gen2 Staging LakehousesMy friend Alex Powers (PM, Fabric CAT) wrote a blog post about cleaning the staging lakehouses generated by Dataflow Gen2. Before reading this blog, go ahead and read his blog first on the mechanics of it and the whys. Note that these are system gene...Discuss·612 readsmssparkutils
StarRocks Engineeringstarrocks.hashnode.dev·May 31, 2024How WeChat’s Lakehouse Design Efficiently Handles Trillions of RecordsAbout WeChat WeChat is the world’s largest standalone mobile app, serving over 1.3 billion monthly active users as a platform for instant messaging, social media, and mobile payments. To support its unprecedented and rapidly expanding user base, WeCh...Discusslakehouse
Stefan StarkeforCloudflight Engineering Blogengineering.cloudflight.io·Feb 14, 2024Lakehouse: Securing data accessIn this article, we'll delve into best practices for securing data access when using Microsoft Fabric Lakehouses. The goal is ensuring that your valuable data remains protected and enable people from within your organization to see subsets of the dat...Discuss·1.0K readsfabric
Sandeep PawarProfabric.guru·Nov 19, 2023Programmatically Creating, Managing Lakehouses in FabricAt MS Ignite, Microsoft unveiled a variety of new APIs designed for working with Fabric items, such as workspaces, Spark jobs, lakehouses, warehouses, ML items, and more. You can find detailed information about these APIs here. These APIs will be cri...Discuss·1 like·5.0K readslakehouse
