Building a Drift Detector, Breaking It on Purpose, and What That Taught MeA few months ago I built a system that watches a live stream of prices and tries to notice when "normal" changes. Not just "the price went up", anyone can see that on a chart, but the harder question:8 min read·dataengineering101.hashnode.devAug 18
Building an Adaptive Anomaly Detection Pipeline for Financial Streams This is not a finished project post. It is a working notes post of what I am building, why I am building it, what is working, what is not yet, and where it is going. I am writing this now, mid-build,7 min read·dataengineering101.hashnode.devJun 22
Build, Break, Ship: What I Learned Building a Real‑Time PipelineI built this pipeline because I wanted to stop talking about streaming systems and start having built one properly. Not a tutorial follow-along. Not a single Kafka consumer writing to a CSV. An actual9 min read·dataengineering101.hashnode.devMay 13
Redis: A Comprehensive Technical Deep DiveWhat Redis Is Redis (Remote Dictionary Server) is an open-source, in-memory data structure store that functions as a database, cache, message broker, and streaming engine. Unlike traditional disk-based databases, Redis keeps its entire dataset in RAM...10 min read·dataengineering101.hashnode.devDec 15, 2025
Distributed File Systems (DFS): Architecture, Use Cases & EvolutionIntroduction Traditional file systems on a single machine just can't keep up with the scale and performance needs of modern data infrastructure. That’s where Distributed File Systems (DFS) come into play. In this blog, I’ll take you through what a DF...5 min read·spark-or-snowflake.hashnode.devNov 30, 2025