Constantin Lungudatawise.dev·11 hours agoCross-dataset Foreign Key referencing in BigQueryIn one of my previous posts I've written about the then-newly-added Primary Key/Foreign Key constraints in BigQuery. While they are not enforced like traditional RDBMS, they can still provide an improvement to query performance. They have one imp...DiscussPractical BigQuerybigquery
Constantin Lungudatawise.dev·12 hours agoUsing STRUCTS for quick analysis in BigQueryI've posted earlier about STRUCTS in BigQuery, here's how I use it from time to time to help me debug and analyze data a bit faster. Since changing filter values for different test cases / observations you are interested about can be a headache (es...DiscussPractical BigQuerySQL
Christopher Garzonchrisgarzon.hashnode.dev·Apr 16, 202410 Best ETL Tools 2024ETL tools automate processes, improve data accuracy, and generate valuable insights. This article reviews the top 10 ETL tools of this year, focusing on their distinctive features, scalability, ease of use, and overall performance. It is intended for...DiscussETL Tools
Christopher Garzonchrisgarzon.hashnode.dev·Apr 16, 2024Data Engineering: Incremental Data Loading StrategiesIncremental data loading is an approach to data integration that transfers only the new or changed records from one database or data source to another, rather than moving the entire data set. This method is especially beneficial in environments where...Discussincremental data loading
Christopher Garzonchrisgarzon.hashnode.dev·Apr 16, 2024Data Engineer Academy ReviewThe Data Engineering Academy takes pride in presenting the transformational story of Vish, who navigated the complexities of a shifting job market to become adept in data engineering, commencing an exciting new chapter in his professional life. Vish...Discusscareer advice
Christopher Garzonchrisgarzon.hashnode.dev·Apr 16, 2024Data Engineer Academy Succes StoriesThe Data Engineering Academy takes pride in presenting the transformational story of Vish, who navigated the complexities of a shifting job market to become adept in data engineering, commencing an exciting new chapter in his professional life. Vish...Discussinterview preparations
Lalit Kaushikdigitalhuts.hashnode.dev·Apr 16, 2024Need More Time? Read These Tips To Eliminate THE SCIENCE OF DATAIntroduction In today's fast-paced world, the role of data science has become increasingly vital in driving innovation, informing decision-making, and optimizing processes across various industries. Businesses are leveraging the power of data to gain...Discusspredictive analytics
Constantin Lungudatawise.dev·Apr 15, 2024Sharded tables in BigQueryHave you ever worked with sharded tables in BigQuery? I've encountered them in a project long time ago and haven't seen them much around since. Does the name not ring a bell? Well, think of it as pseudo-partitioning, a way to store data split bet...Discuss·46 readsPractical BigQuerybigquery
Diptarup Mukherjeediptarup.hashnode.dev·Apr 14, 2024Optimizing SQL query performance:In order to understand the optimization of SQL queries first we need to understand what actually happens in the background when we query our database table. So grab a cup of coffee and lets start our discussion by considering a scenario. All query ex...DiscussDatabases
Constantin Lungudatawise.dev·Apr 11, 2024Comparing tables with FULL OUTER JOINDoes your Data Engineering project use a data-diffing tool? Say you're preparing to deploy a change to a prod table. You've changed the way some metrics are calculated and twisted some filters. How do you find out what's different between two table...DiscussPractical BigQuerySQL
