Omprakash Sridharanomprakashsridharan.hashnode.dev·Jun 24, 2023Rust multi module microservices Part 6 - Books APII hope you've reached this point, and now we'll begin creating our first microservice application: a books HTTP REST API with a single POST endpoint for adding a book. However, before we proceed, we must establish a migration setup for this service t...DiscussRust super appRust
Omprakash Sridharanomprakashsridharan.hashnode.dev·Jun 24, 2023Rust multi module microservices Part 5 - CommonEvery workspace can have multiple application crates that can have some common utility functions, constants, models/structs for HTTP Request/Response, Kafka Messages, and much more. For this purpose, we are having the common crate to be used for hold...DiscussRust super appRust
Omprakash Sridharanomprakashsridharan.hashnode.dev·Jun 24, 2023Rust multi module microservices Part 2 - Workspace dependenciesBefore we delve into the sub-crate modules, let's add a few crates that are common to the workspace and can be shared among all the crates. Update your Cargo.toml with the dependencies and it should look like the below. [workspace] members = ["common...DiscussRust super appRust
Adam Zhangfordata_with_adam_blogdatawithadam.hashnode.dev·Jun 14, 2023Apache Airflow Demystify - 1 - basicsWhat is Airflow Apache Airflow is a popular open-source orchestrating tools for authoring, scheduling and monitoring data pipeline/workflow. It has some modern data tech stack features such as It is a workflow as a code that allows you to manage you...Discussdataengineering
Rohan Anandrohan-anand.hashnode.dev·May 18, 2023Improving CLI Usability in AirflowIntroduction: In the world of open source, I recently had an exciting opportunity to contribute to the Airflow project, an essential tool for orchestrating and scheduling data workflows. During my contribution, I encountered an issue related to the -...Discuss·38 readsapache-airflow
Warui Wanjiruwaruithemystery.hashnode.dev·May 5, 2023Orchestration? Maybe! (Project Beat~lytica Part 3)Welcome to the third instalment of the Beatlytica series. I'm sorry to have kept you waiting for the juice this week. It's been a little hectic, but we're moving. In the last instalment, we discussed the implementation of the consumer of messages fro...DiscussProject Beatlyticadata-engineering
Arun R Nairarunrnair.hashnode.dev·May 4, 2023Mage.ai vs Apache Airflow⭕ Apache Airflow and Mage are two popular tools used in data engineering and workflow automation. While both tools share some similarities, they have distinct differences that set them apart. ⭕ Apache Airflow is an open-source platform for creating...Discuss·648 readsETL
Karl Bolingerkbolinger.hashnode.dev·Apr 17, 2023Building Data Pipelines with Apache Airflow: A Complete Guide with ExamplesData pipelines are a critical component of modern data infrastructures, allowing organizations to efficiently manage and process large volumes of data. Apache Airflow is an open-source platform that helps developers to create and manage data pipeline...Discuss·29 readsData Engineering Basicsdata-engineering
Andrew Sharifikiaalireza-sharifikia.hashnode.dev·Mar 17, 2023DataOps: Apache Airflow - BasicIntroduction Apache Airflow is an open-source platform for authoring, scheduling, and monitoring data and computing workflows. It was developed by Airbnb and is now under the Apache Software Foundation.It uses Python to create workflows that can be e...Discuss·133 readsDataOpsairflow
Arnold Kamanziblog.k-arnold.com·Mar 3, 2023Automate DB Backup with Apache AirflowIf like me, you have administered a database and always find yourself tired of running the scripts to back up the database or even worse, you use a cronjob to automate the backup process, then something goes wrong and suddenly the backup fails and yo...Discuss·156 readsapache-airflow
