NHNweke Henryinhenrycloud.hashnode.dev·Jul 14 · 8 min read My Alert Fired at 4:40am. Three Places Were Notified Before I Woke Up. (Week 5)Here is what happened while I was asleep. CPU hit 99.85% on my node at 4:40am. My terminal printed the alert. Slack sent a message to #observability-critical. My Gmail received an email. Five minutes 00
NHNweke Henryinhenrycloud.hashnode.dev·Jul 1 · 7 min readI Found a Payment Failure Hidden in 600 Log Lines in 4 Seconds. Here's How. (Week 4)Here's a scenario every engineer has lived through. Your Prometheus alert fires at 3am. Error rate spiked to 17%. You stare at the metric graph. The line goes up, stays up, comes down. You know someth00
NHNweke Henryinhenrycloud.hashnode.dev·Apr 28 · 4 min readHow I Built a Real-Time DDoS Detection Engine from ScratchIntroduction When your web application is under attack, every second counts. In this post I'll walk you through how I built a real-time anomaly detection engine that watches HTTP traffic, learns what 00
NHNweke Henryinhenrycloud.hashnode.dev·Apr 12 · 10 min readGrafana Dashboards as CodeGrafana Dashboards as Code — Destroy the Container, the Dashboard Survives (Week 3) Someone commented on my last post: "Don't forget the nice gradient shade, give it some bling." Challenge accepted. B00
NHNweke Henryinhenrycloud.hashnode.dev·Mar 30 · 9 min readPrometheus Deep Dive — Alerts, Debugging, and What Actually Happens When Things Don't Work (Week 2) Last week I spun up a working observability stack from scratch. This week I went deeper. And I hit real problems. I'm documenting both — because the de00