Working mostly on Data Engineering, Machine Learning and Bayesian analytics.
Nothing here yet.
Jan 29, 2025 · 8 min read · “But how do you ensure every BigQuery table you create has the same metadata in Dataform?” Yes, I thought to myself, how do I ensure that? This was a question posed recently to me and I realized I don’t quite have an answer. And I really didn’t want ...
Join discussion
Nov 24, 2024 · 8 min read · Reliable, accurate data is the foundation of data-driven decision-making. Poor data quality can lead to incorrect insights, a lack of trust in data and data products. It really is quite an obvious statement to make, but what’s not obvious is how exac...
Join discussion
Nov 4, 2024 · 9 min read · Dataform is a tool that creates data pipelines using SQL. If you’re familiar with Dbt, Dataform is probably best understood as Dbt-esque tool that integrates really well with BigQuery and other Google Cloud products. In a short amount of time, it’s q...
Join discussion
Oct 17, 2024 · 2 min read · Sometimes it so happens that we create or are using a table with data that is non-partitioned but we need to convert this into a partitioned table. A typical use-case is old tables that start accumulate data over time. Quite often, we need the same d...
Join discussion
Mar 24, 2024 · 6 min read · Cloud Run helps you deploy containerized workloads at scale. Using it as a backend for web server use cases is quite well known, but it also can be very useful for large batch jobs that tend to be CPU heavy, especially if the job can be divided into ...
Join discussion