Constantin Lungudatawise.dev·Feb 13, 2025How ARRAY() can function as UNPIVOT and UNNEST as PIVOT?I’ve come across this SQL transformation multiple times, and it’s an interesting two-way problem. 1️⃣ From columns to rows (ARRAY as UNPIVOT):We start with separate timestamps for different lifecycle events. To analyze events dynamically, we reshape ...Practical SQLbigquery
Dineshdinesh-dezoomcamp.hashnode.dev·Feb 12, 2025My Journey Through BigQuery: Lessons from Data Engineering Zoomcamp Module 3As I wrap up Module 3 of the Data Engineering Zoomcamp, I want to share my experience working with Google BigQuery and the valuable insights I gained while analyzing the NYC Yellow Taxi dataset. This module has fundamentally changed how I think about...#dezoomcamp
Constantin Lungudatawise.dev·Feb 12, 2025Choosing the Right Ranking Function: Why Ties in SQL MatterIf you’re like me, you probably use QUALIFY + ROW_NUMBER() almost daily for deduplication or finding the first/last occurrence of something. It’s a powerful combo in modern SQL ! But here’s the catch: there are subtle nuances and edge cases that can ...Practical SQLbigquery
Constantin Lungudatawise.dev·Feb 8, 2025A closer look at STRING_AGG in BigQueryModern SQL engines have a wealth of aggregation functions. Here's a quick example that makes use ofBigQuery STRING_AGG. What does it do? It aggregates all the values in a grouping, joined by a separator of our choice, creating a string of those...Practical SQLbigquery
Constantin Lungudatawise.dev·Feb 8, 2025Here's a great use case for GenAI writing SQLTruth be told, I almost never use GenAI as part of my day-to-day work as a Data Engineer. For the most part, it’s just easier and faster for me to write the SQL code myself. There are a few reasons for this:➡️ If the task is complex enough to need Ch...Practical SQLSQL
Constantin Lungudatawise.dev·Feb 8, 2025Using EXISTS with LOGICAL_OR in BigQueryLong time, no see! Here's a quick SQL exercise that illustrates some important modern concepts. So, we're given a list of updates per each order, and at each point in time we have some flags. Our goal here is to check for each order if there was any ...Practical SQLbigquery
PAVAN RAYABAGIpipelinewizard.hashnode.dev·Jan 10, 2025Automating Python Script for Google BigQuery using CrontabAutomation is a powerful tool that saves time and ensures consistency, especially for repetitive tasks like data operations. This guide will walk you through automating the execution of a Python script for Google BigQuery using crontab on a Linux sys...Automation Python Google BigQuery Crontab Linux Data Engineering Task Scheduling Cloud Google Cloud Platform (GCP) Shell Scripting Big Data Python Automation Dev
Fredrick Simifredricksimi.hashnode.dev·Jan 9, 2025Day 3 - Ingesting NBA Player Data into BigQuery using Python, Cloud Storage, and the Sportsdata.io APIIn this article, we will learn how we can fetch data from the Sportsdata.io API, store that data in Google Cloud Storage & load that data into a BigQuery table. The code for this article can be found in this GitHub repo under the Day_3 folder. Prereq...30 Days of DevOps - 2025Cloud Computing
Japkeerat Singhjapkeeratsingh.com·Dec 26, 2024What I learned about PyPi from maintaining an Open-Source PackageLast month, I published a package on PyPi - TezzCrawler - a simple CLI tool to convert any website to LLM ready draft for building a RAG capabilities on any website. What spiraled next, was hundreds of hours of analysis on how PyPi works and what hap...17 likes·29 readsOnlySubspypi
Kelly Gloriaedutechno.hashnode.dev·Dec 24, 2024Tableau Training: Key Features, Capabilities, and Learning PathsIn the current world of data-driven companies, they are always looking for tools to transform complicated data into useful insight. Tableau has been recognized as an extremely efficient and widely used software for visualization of data. With its use...tableau training
