Kafka Consumer Container Restarts in Kubernetes: A Production Case Study

Background At my current org, I got introduced to a very fun tool, PagerDuty - because who doesn't love being woken up at 3:31 AM? A similar alert flared up for one of our merchant services. While it mercifully avoided the 3 AM slot, the diagnostic w...