FFFred Fenginfredfeng.hashnode.dev·1d ago · 6 min readCronflower: Run a DAG workflow across your cluster, instead of chaining cron jobscronflower is an open-source, distributed scheduler for the JVM with a web console. cronflow is its optional DAG add-on: declare the steps and how they depend on each other as a graph, and the same cl00
TSTao Sunintech-mind.hashnode.dev·Aug 15 · 9 min readHow I Built a Secure AI Database Gateway in Go Using MCP, Anthropic, and KafkaBuilt DynamoDB Sage: a Go MCP server that lets LLM agents safely query and manage Amazon DynamoDB via natural language A custom RiskAnalyzer validates every mutating or heavy tool call before it touc00
ATAndrew Taninlayline.hashnode.dev·Jul 31 · 9 min readYour Data Warehouse Is Not Your Data PipelineTeams keep forcing their warehouse to do integration work it was never designed for. The result is ballooning costs, opaque failures, and architectures that become harder to maintain the more they ‘su00
KDKarthik Darbhaintech4nirvana.com·Jul 30 · 8 min readObservability That Thinks: AI for Pipeline MonitoringThe Problem Most pipeline observability today is a wall of thresholds. Row count dropped below X. Job ran longer than Y minutes. Null percentage exceeded Z. Someone picked those numbers months ago, of43JKN
COcharles onokohwomoincharlesonokohwomo.hashnode.dev·Jul 14 · 12 min readPhase 2B — Building the ADIP AI Insight EngineSeries: ADIP Engineering Intelligence Phase 2B — AI Insight Generation Layer Repository: github.com/CKohwo/ADIP-Intelligence-lab Introduction Phase 2A gave ADIP discipline. It built the data foundatio00
ISIkraj Singhinikrajsingh.hashnode.dev·Jul 13 · 10 min readData Pipelines in the Age of GenAIThis article is written in my personal capacity. The views expressed are my own and do not represent Amazon or my employer. Examples are based on public information and synthetic scenarios; no confide00
HTHarsh Trivediinharshtrivedii.hashnode.dev·Jul 11 · 20 min readBuilding a Secure SharePoint → Azure Blob → Snowflake Document Intelligence PipelinePart of the series AI Cloud and Data Engineering Articles. In Part 1 we covered why documents should be encrypted before they ever reach cloud storage, how Azure Key Vault and Service Principals keep 00
SGSergio González Téllezinevankhandev.hashnode.dev·Jul 2 · 2 min readWhy do so many systems appear to produce new information when, in reality, they only reorganize existing information?SEED-003 PROBLEM Why do so many systems appear to produce new information when, in reality, they only reorganize existing information? INSIGHT Data is rarely created; it is usually transformed, filt00
JLJeremy Longshoreinjeremylongshore.hashnode.dev·Jul 1 · 4 min readSTCI Zero to v0.1.0: A Token Cost Index in One DayYesterday's ADR said what STCI would be. Today it exists. Eleven commits. One repo. Full pipeline from data collection to production API, released as v0.1.0 by end of day. The Pipeline STCI — the toke00
DDatawinderindatawinder.hashnode.dev·Jun 10 · 9 min readBuilding a Lean, Single-Worker Broken URL Monitor for Data PipelinesThe Technical Problem: Websites Drift, Pipelines Don't Know Long-running scraping pipelines have a structural assumption baked in: the URLs you configured last month still resolve today. That assumpti10