Anand KumarforAltimate Team Blogblog.altimate.ai·Apr 14, 2024How I Cut dbt Development Time by 20% with the Power User Vscode ExtensionWorking with data in any capacity—be it as a data/analytics engineer, data scientist, or data analyst—inevitably dealing with the need for documentation. Whether it's clarifying unclear fields, understanding dataset structures, or seeking explanation...Discuss·3 likes·83 readsdbt
Constantin Lungudatawise.dev·Apr 5, 2024A portable Data Analytics stack using Docker, Mage, dbt-core, DuckDB and SupersetJust wanted to share a small learning-by-doing project of mine. It's a containerized Data Analytics suite, covering end-to-end analytics process for a small (imaginary) company. We're talking about:- generating example data in parquet files using Pyt...Discuss·60 readsLearning JourneyDocker
Jesper Baggefullouterjoin.dev·Mar 19, 2024Automagic schema inference killed the data engineerTodays lesson in working with legacy output is: data contracts, data contracts and data contracts. The great thing about shipping structured data as text files is it is so darn easy! Just store it as a CSV and boom: minimum overhead and just the righ...Discuss·30 readsClickHouse
ZamboforDataChef's Blogblog.datachef.co·Feb 12, 2024Speed up dbt workflow with TaskDo you remember the last time you opened your laptop and thought: "I can't wait to spend half my day writing configs in YAML."? Yeah, neither do I. But if you use dbt heavily, I'm afraid you and YAML are in for the long run. Even with the help of dbt...Discuss·308 readssoftware development
Brice Luubluu.dev·Feb 7, 2024Modern Data Modeling: Start with the End?Introducing: dbt So what's this dbt everyone's fussing about?? At its core, dbt (data build tool) is a "modern" data modeling framework using (Jinja) templated SQL. It comes with a CLI tool that allows to easily materialize your models as tables or v...Discuss·33 readsdata
Mohammed Alhakeemalhakeem.me·Jan 31, 2024Building robust data pipelines with Automated DDL changes handling using Snowflake and dbtIn most scenarios, the data engineering team creates processes that import data in batches from transactional database sources into a central analytical database. We also usually keep the history of such tables by implementing SCD-2 to keep track of ...Discuss·34 readsdbt
Aditya Tiwaridataml.hashnode.dev·Nov 16, 2023Modeling and Transforming data for analytics using SQL, Dbt and AWS Redshift.Introduction I'll be going through a hypothetical scenario at an event ticketing company and describe the data modeling and transformation process in the STAR format. The github repository of the project is linked below.(https://github.com/Aditya-Tiw...Discuss·10 likes·355 readsdata-engineering
Constantin Lungudatawise.dev·Aug 16, 2023A portable data stack with Dagster, Docker, DuckDB, dbt and SupersetI've previously mentioned how pet projects are good for exploring new technologies. It's not every day that you can work on a greenfield project with just the stack you want. So while ago I decided to spin off a previous proof of concept I've had (po...Discuss·1 like·910 readsLearning Journeydbt
Rahul Bhaverahulbhave.hashnode.dev·Jul 30, 2023Automating PostgreSQL Database Creation, dbt run and running tests using GitHub ActionsIntroduction: In today's fast-paced development environment, automating repetitive tasks like database creation and testing is crucial for ensuring a smooth and efficient workflow. GitHub Actions provides a powerful platform for automating various ta...Discuss·41 readsdbt
Adam Zhangfordata_with_adam_blogdatawithadam.hashnode.dev·Jul 28, 2023ONE dbt project for multiple environmentsMotivation The goal of this blog is to show how to use dbt to manage multiple database environments for data engineers or analysts. Due to the delay in sync between QA, development and production database, your SQL code might be slightly different fo...Discuss·33 readsdbt
