Feb 12 · 11 min read · Why Traditional Airflow DAG Patterns Fail at Scale Many data engineers approach Airflow DAG design with patterns borrowed from traditional ETL tools or batch processing frameworks that assume linear, one-time execution. These patterns break down cata...
Join discussionAug 31, 2025 · 7 min read · How Sui's Mysticeti Protocol Achieves 390ms Latency and Scales to Handle Massive Throughput The Current State of Blockchain Consensus The blockchain industry continues to navigate the fundamental trade-off between decentralization, security, and scal...
Join discussion
Jun 20, 2025 · 8 min read · Introduction to DAGs: One-Way Paths in the Colony Imagine an ant colony where some tunnels are one-way paths. Once an ant travels down these tunnels, there’s no turning back—they can only keep moving forward through specific chambers. These one-way p...
Join discussion
Jun 8, 2025 · 16 min read · Alright, let's talk Kubernetes. It's powerful, no doubt. For managing containerized applications, scaling services, and ensuring high availability, it's become the de facto standard for many of us. I've used it extensively, and for long-running servi...
Join discussionJun 9, 2025 · 4 min read · In the world of modern data engineering, workflow automation is the key to managing and scaling complex processes. Whether you're scheduling ETL jobs, automating data ingestion, or orchestrating machine learning pipelines, Apache Airflow has become o...
Join discussion
Feb 10, 2025 · 5 min read · Tip 1 : Register the Microsoft.DesktopVirtualization Before using AVD make sure to register it. In the PowerShell session in the Azure Cloud Shell pane, run the following command to register the Microsoft.DesktopVirtualization resource provider: Regi...
Join discussionJan 11, 2025 · 5 min read · Introduction Topological sorting is a fundamental concept in graph theory, used to order the vertices of a directed acyclic graph (DAG) linearly. It ensures that for every directed edge u→vu \to vu→v, vertex uuu appears before vertex vvv in the order...
Join discussion
Dec 27, 2024 · 4 min read · Crypto Insights Post-Quantum Cryptography enhanced Bitcoin Fork Quantum computing poses a potential threat to encryption methods like Bitcoin’s elliptic curve cryptography (ECC). The project, pqcBitcoin (GitHub repo), integrates post-quantum cryptogr...
Join discussion
Oct 5, 2024 · 15 min read · Embrace new formats and experiences; they bring more voices to the conversation. Hello everyone! I am happy to kick off our 6th session. It's actually the second one for this year and the first in Telegram. This is why we are experiencing a little bi...
Join discussionAug 26, 2024 · 3 min read · Apache Spark's execution model is one of the reasons it stands out as a powerful tool for big data processing. At its core, Spark's execution model revolves around two main concepts: transformations and actions. To understand how Spark operates, it’s...
Join discussion