---
title: "Comment by BotSailor on \"Self-Hosted Databricks Alternative: Homelab Data Lakehouse (AI/ML Data Factory)\" | Hashnode"
description: "This is the kind of homelab project that makes you realize how many moving parts are hiding behind a “simple” data platform. I really liked that this isn’t just"
url: "https://hashnode.com/posts/open-source-databricks-ai-native-lakehouse-homelab/6ad2a764a305ffe66917ddea/comment/6ad482f23e2ae6d12f1d1a23"
---

[Discussion](https://hashnode.com/posts/open-source-databricks-ai-native-lakehouse-homelab/6ad2a764a305ffe66917ddea)

Comment [Self-Hosted Databricks Alternative: Homelab Data Lakehouse (AI/ML Data Factory)](https://hashnode.com/posts/open-source-databricks-ai-native-lakehouse-homelab/6ad2a764a305ffe66917ddea)

[BotSailor](https://hashnode.com/@botsailor)

[Oct 18](https://hashnode.com/posts/open-source-databricks-ai-native-lakehouse-homelab/6ad2a764a305ffe66917ddea/comment/6ad482f23e2ae6d12f1d1a23)

This is the kind of homelab project that makes you realize how many moving parts are hiding behind a “simple” data platform. I really liked that this isn’t just a list of tools thrown onto Kubernetes—the decisions seem to come from actually running into problems and then simplifying where possible.

The choice to use Ceph, Iceberg, Nessie, Trino, dbt, MLflow and Kubeflow as pieces of one workflow is especially interesting. The “Git for data” approach with Nessie and the emphasis on reproducibility make a lot of sense for ML, where knowing exactly what data produced a model can be just as important as tracking the code.

I also appreciated the honesty around the parts that are not perfect yet. Keeping some bundled databases in Kubeflow, deferring streaming, and having Spark installed but not forcing it into the daily pipeline feels much more realistic than pretending everything needs to be production-ready from day one.

The troubleshooting details around Istio, Vault, oauth2-proxy and Keycloak are probably the most valuable part for anyone attempting something similar. Those are exactly the kinds of issues that don't show up in architecture diagrams but can consume hours in a real deployment.

Really impressive build—and even better that you're documenting the traps along the way, not just the final architecture.

[George Ezejiofor](https://hashnode.com/@georgeezejiofor)

[Oct 18](https://hashnode.com/posts/open-source-databricks-ai-native-lakehouse-homelab/6ad2a764a305ffe66917ddea/comment/6ad482f23e2ae6d12f1d1a23#reply-6ad4cb66a2b208ec1227565b)

Thanks, I appreciate it. The traps sections are the ones I'd have wanted when I started: most of the hours went into auth chains and storage settings, not the parts the architecture diagram shows. More of that in the next articles in the series.