Combine Dataform with Typescript for Improved Workflows“But how do you ensure every BigQuery table you create has the same metadata in Dataform?” Yes, I thought to myself, how do I ensure that? This was a question posed recently to me and I realized I don’t quite have an answer. And I really didn’t want ...Jan 29, 2025 · 8 min readraghuveer.me
Writing data quality tests in DataformReliable, accurate data is the foundation of data-driven decision-making. Poor data quality can lead to incorrect insights, a lack of trust in data and data products. It really is quite an obvious statement to make, but what’s not obvious is how exac...Nov 24, 2024 · 8 min readraghuveer.me
A Practical Introduction to Google Cloud DataformDataform is a tool that creates data pipelines using SQL. If you’re familiar with Dbt, Dataform is probably best understood as Dbt-esque tool that integrates really well with BigQuery and other Google Cloud products. In a short amount of time, it’s q...Nov 4, 2024 · 9 min readraghuveer.me
Partition and cluster an existing BigQuery tableSometimes it so happens that we create or are using a table with data that is non-partitioned but we need to convert this into a partitioned table. A typical use-case is old tables that start accumulate data over time. Quite often, we need the same d...Oct 17, 2024 · 2 min readraghuveer.me
Using Google Cloud Run to process batch jobsCloud Run helps you deploy containerized workloads at scale. Using it as a backend for web server use cases is quite well known, but it also can be very useful for large batch jobs that tend to be CPU heavy, especially if the job can be divided into ...Mar 24, 2024 · 6 min readraghuveer.me
