Constantin Lungudatawise.dev·Feb 8, 2025A closer look at STRING_AGG in BigQueryModern SQL engines have a wealth of aggregation functions. Here's a quick example that makes use ofBigQuery STRING_AGG. What does it do? It aggregates all the values in a grouping, joined by a separator of our choice, creating a string of those...Practical SQLbigquery
Constantin Lungudatawise.dev·Feb 8, 2025Here's a great use case for GenAI writing SQLTruth be told, I almost never use GenAI as part of my day-to-day work as a Data Engineer. For the most part, it’s just easier and faster for me to write the SQL code myself. There are a few reasons for this:➡️ If the task is complex enough to need Ch...Practical SQLSQL
Constantin Lungudatawise.dev·Feb 8, 2025Using EXISTS with LOGICAL_OR in BigQueryLong time, no see! Here's a quick SQL exercise that illustrates some important modern concepts. So, we're given a list of updates per each order, and at each point in time we have some flags. Our goal here is to check for each order if there was any ...Practical SQLbigquery
PAVAN RAYABAGIpipelinewizard.hashnode.dev·Jan 10, 2025Automating Python Script for Google BigQuery using CrontabAutomation is a powerful tool that saves time and ensures consistency, especially for repetitive tasks like data operations. This guide will walk you through automating the execution of a Python script for Google BigQuery using crontab on a Linux sys...Automation Python Google BigQuery Crontab Linux Data Engineering Task Scheduling Cloud Google Cloud Platform (GCP) Shell Scripting Big Data Python Automation Dev
Fredrick Simifredricksimi.hashnode.dev·Jan 9, 2025Day 3 - Ingesting NBA Player Data into BigQuery using Python, Cloud Storage, and the Sportsdata.io APIIn this article, we will learn how we can fetch data from the Sportsdata.io API, store that data in Google Cloud Storage & load that data into a BigQuery table. The code for this article can be found in this GitHub repo under the Day_3 folder. Prereq...30 Days of DevOps - 2025Cloud Computing
Japkeerat Singhjapkeeratsingh.com·Dec 26, 2024What I learned about PyPi from maintaining an Open-Source PackageLast month, I published a package on PyPi - TezzCrawler - a simple CLI tool to convert any website to LLM ready draft for building a RAG capabilities on any website. What spiraled next, was hundreds of hours of analysis on how PyPi works and what hap...17 likes·27 readsOnlySubspypi
Kelly Gloriaedutechno.hashnode.dev·Dec 24, 2024Tableau Training: Key Features, Capabilities, and Learning PathsIn the current world of data-driven companies, they are always looking for tools to transform complicated data into useful insight. Tableau has been recognized as an extremely efficient and widely used software for visualization of data. With its use...tableau training
Ambrus Pethesmitzu.hashnode.dev·Dec 20, 2024Top 5 self-service BI solutions for BigQueryWhat is Google BigQuery? Google BigQuery is a cloud service for processing large datasets, offering fast SQL-like queries and analytics. With its powerful architecture, BigQuery allows you to execute complex queries on massive datasets, ranging from ...bigquery
Pieterjan Lambeinserver-side-stories.hashnode.dev·Dec 9, 2024Unlocking Savings: Mastering BigQuery's Capacity-Based Compute Pricing ModelTL;DR BigQuery has two pricing models: On-demand, charging per byte processed, and Capacity-based, charging for slot capacity over time. Capacity-based can be cheaper if monthly costs are high. It involves reserving slots and can be optimized with hi...bigquery
Vaibhav Singhvaibhavcode.hashnode.dev·Dec 1, 2024Structured data prediction using Vertex AI PlatformIn this post, we delve into how to create a robust workflow for predicting a baby’s weight using structured data and Google Cloud's Vertex AI Platform. From data preparation to deploying a machine learning (ML) model, we’ll explore the process step b...Machine Learning