bobblackburn.comAzure Data Factory CI/CD with GitHub and Azure PipelinesWithout source control for Azure Data Factory (ADF), you only have the option to publish your pipeline. And, it has to validate. Now with source control, we can save intermediate work, use branches, and publish when we are ready. The next step is CI/...Aug 19, 2024·6 min read
bobblackburn.comCI/CD with GitHub Database Projects and Azure PipelinesWith the high pace of development, data architects/engineers have to be able to keep up with our DevOps counterparts. We also want the many benefits of CI/CD for databases such as fewer errors, stable builds, better release management, and less manua...May 19, 2024·6 min read
bobblackburn.comDatabricks Zip File ValidationIn many IoT systems, you will receive feeds that are in a zip format. Sometimes they are empty. We want to validate they contain a file before we start processing them because it involves moving the file around different file systems, running a shell...Apr 29, 2024·2 min read
bobblackburn.com3 Steps to Run PowerShell in Azure Data FactoryAzure Data Factory has many capabilities. But no tool is the best at everything. Sometimes you have an existing script that needs to be automated or PowerShell is the best programming option for the task at hand. Currently, ADF does not have a PowerS...Apr 29, 2024·3 min read
bobblackburn.comSQL Server Indexing: 5 Steps to Better PerformanceCustomers don’t want to wait more than a few seconds for an application to process and move to the next step. And we keep getting squeezed on our batch window to do all our daily processing. We cannot just throw more hardware at the problem. It may b...Apr 29, 2024·6 min read
