Oct 15, 2024 · 8 min read · Imagine managing a large-scale data environment where you have tens of different database servers spread across multiple regions or environments, including production, development, and staging. Each of these database servers requires its own set of c...
Join discussion
Aug 19, 2024 · 6 min read · Without source control for Azure Data Factory (ADF), you only have the option to publish your pipeline. And, it has to validate. Now with source control, we can save intermediate work, use branches, and publish when we are ready. The next step is CI/...
Join discussion
Jul 19, 2024 · 1 min read · I was building a data factory source the other day and I needed to skip the first 5 rows. This is intuitive and easy within a text (csv) source as there is a setting to skip the first N rows. In excel there is a 'Range' option which has as an example...
Join discussion
Apr 29, 2024 · 3 min read · Azure Data Factory has many capabilities. But no tool is the best at everything. Sometimes you have an existing script that needs to be automated or PowerShell is the best programming option for the task at hand. Currently, ADF does not have a PowerS...
Join discussion
Oct 3, 2023 · 2 min read · To understand Azure Key vaults, think of a scenario like a backend connection to databases. The question is “Do we directly use connection strings in our code?”. Storing connection strings and other sensitive information directly in your code is gene...
Join discussion
Aug 7, 2023 · 7 min read · In all of my Fabric related blog posts, I make it abundantly clear that Fabric is (as of the writing of the post), in public preview and the performance/features may change at any time. I want to discuss one such significant performance improvement i...
Join discussion