Securing Unity Catalog - Storage Credentials, Scoped Service Principal External Location access - serverless autoloader file notification file events

Jan 18 · 4 min read · One of the most common challenges in Data Engineering is giving a pipeline access to just enough data without handing over the keys to the entire kingdom (or S3 bucket). In this post, we’ll walk throu