Ambrus Pethesmitzu.hashnode.dev·Dec 20, 2024Top 5 self-service BI solutions for BigQueryWhat is Google BigQuery? Google BigQuery is a cloud service for processing large datasets, offering fast SQL-like queries and analytics. With its powerful architecture, BigQuery allows you to execute complex queries on massive datasets, ranging from ...bigquery
Pieterjan Lambeinserver-side-stories.hashnode.dev·Dec 9, 2024Unlocking Savings: Mastering BigQuery's Capacity-Based Compute Pricing ModelTL;DR BigQuery has two pricing models: On-demand, charging per byte processed, and Capacity-based, charging for slot capacity over time. Capacity-based can be cheaper if monthly costs are high. It involves reserving slots and can be optimized with hi...bigquery
Vaibhav Singhvaibhavcode.hashnode.dev·Dec 1, 2024Structured data prediction using Vertex AI PlatformIn this post, we delve into how to create a robust workflow for predicting a baby’s weight using structured data and Google Cloud's Vertex AI Platform. From data preparation to deploying a machine learning (ML) model, we’ll explore the process step b...Machine Learning
Raghuveer Sriramanraghuveer.me·Nov 24, 2024Writing data quality tests in DataformReliable, accurate data is the foundation of data-driven decision-making. Poor data quality can lead to incorrect insights, a lack of trust in data and data products. It really is quite an obvious statement to make, but what’s not obvious is how exac...Practical guide to build data pipelines with Dataformdataform
Raghuveer Sriramanraghuveer.me·Nov 4, 2024A Practical Introduction to Google Cloud DataformDataform is a tool that creates data pipelines using SQL. If you’re familiar with Dbt, Dataform is probably best understood as Dbt-esque tool that integrates really well with BigQuery and other Google Cloud products. In a short amount of time, it’s q...4 likes·62 readsPractical guide to build data pipelines with Dataformdataform
Constantin Lungudatawise.dev·Nov 1, 2024Sometimes, you have to use subqueries!Query without FROM clause cannot have a WHERE clause, goes the old SQL adage. So I had this interesting problem the other day. Let's say an order has three boolean flags, each indicating whether a particular error has occurred during its lifetime. Ou...43 readsPractical SQLbigquery
Constantin Lungudatawise.dev·Oct 27, 2024Not all NULLS are the sameSo NULLs are definitely beasts of their own and as Data Engineers we come to learn to take them into account. That is because not knowing their quirks can lead to unexpected results or errors. Let's look at how not all NULLS are the same in BigQuery ...37 readsPractical SQLbigquery
Constantin Lungudatawise.dev·Oct 27, 2024Extracting keys from JSON in BigQueryA couple of months ago, I've posted about dynamically extracting key-value pairs from JSON in BigQuery SQL which leveraged regex (check comments). Shortly after that post, we've gotten a new built-in function to dynamically extract the keys occurring...51 readsPractical SQLbigquery
Constantin Lungudatawise.dev·Oct 21, 2024Computing a hash aggregation in BigQuerySo I've seen Snowflake has an HASH_AGG function. When would we need it? Every time we'd like to work out if ANY value in a group (or the entire table) has changed in any way, even a single extra blank space. While BigQuery does not have it yet, we ca...29 readsPractical SQLbigquery
Constantin Lungudatawise.dev·Oct 20, 2024Another look at ANY_VALUE in BigQueryA reminder that ANY_VALUE is a pretty interesting aggregation function in BigQuery SQL. It gives you a chosen row from a group. Chosen doesn't mean random, but non-deterministic. Together with HAVING MAX | MIN you can actually control what rows get p...39 readsPractical SQLbigquery
