Oncall Adventures - When your Prometheus-Server mounted to GCE Persistent Disk on K8s is Full

Note to anyone that lands on this page in the middle of an Incident and just ✨ needs the solution ✨ . Problem: Prometheus-server running in K8s on GCP using a Persistent Volume has run out of disk. Symptoms: Grafana shows readings diving off cliff. ...