This is the kind of homelab project that makes you realize how many moving parts are hiding behind a “simple” data platform. I really liked that this isn’t just a list of tools thrown onto Kubernetes—the decisions seem to come from actually running into problems and then simplifying where possible.

The choice to use Ceph, Iceberg, Nessie, Trino, dbt, MLflow and Kubeflow as pieces of one workflow is especially interesting. The “Git for data” approach with Nessie and the emphasis on reproducibility make a lot of sense for ML, where knowing exactly what data produced a model can be just as important as tracking the code.

I also appreciated the honesty around the parts that are not perfect yet. Keeping some bundled databases in Kubeflow, deferring streaming, and having Spark installed but not forcing it into the daily pipeline feels much more realistic than pretending everything needs to be production-ready from day one.

The troubleshooting details around Istio, Vault, oauth2-proxy and Keycloak are probably the most valuable part for anyone attempting something similar. Those are exactly the kinds of issues that don't show up in architecture diagrams but can consume hours in a real deployment.

Really impressive build—and even better that you're documenting the traps along the way, not just the final architecture.