---
title: "Comment by Rahul R on \"Self-Hosted Databricks Alternative: Homelab Data Lakehouse (AI/ML Data Factory)\" | Hashnode"
description: "The dbt + Apache Iceberg + Project Nessie workflow. When a dbt build fails halfway through, how do you ensure that temporary Iceberg files and snapshots created"
url: "https://hashnode.com/posts/open-source-databricks-ai-native-lakehouse-homelab/6ad2a764a305ffe66917ddea/comment/6ad323404113334674537eb3"
---

[Discussion](https://hashnode.com/posts/open-source-databricks-ai-native-lakehouse-homelab/6ad2a764a305ffe66917ddea)

Comment [Self-Hosted Databricks Alternative: Homelab Data Lakehouse (AI/ML Data Factory)](https://hashnode.com/posts/open-source-databricks-ai-native-lakehouse-homelab/6ad2a764a305ffe66917ddea)

[Rahul R](https://hashnode.com/@rahul-data)

[Oct 17](https://hashnode.com/posts/open-source-databricks-ai-native-lakehouse-homelab/6ad2a764a305ffe66917ddea/comment/6ad323404113334674537eb3)

The dbt + Apache Iceberg + Project Nessie workflow. When a dbt build fails halfway through, how do you ensure that temporary Iceberg files and snapshots created on the dbt-audit branch are cleaned up safely without affecting tables referenced by existing Nessie tags or MLflow models? Would you recommend Nessie garbage collection with a retention policy, or is there another approach for handling this in production?

[George Ezejiofor](https://hashnode.com/@georgeezejiofor)

[Oct 17](https://hashnode.com/posts/open-source-databricks-ai-native-lakehouse-homelab/6ad2a764a305ffe66917ddea/comment/6ad323404113334674537eb3#reply-6ad3cc3d95422bf5f1f6d896)

Good question. Two things keep it safe here: the failed build never touches main, and nothing deletes files directly.

dbt runs write-audit-publish. Each run first points the dbt-audit branch at main's head, runs dbt build against that branch, and merges to main only if the whole build passes. If it fails halfway, main is untouched, and whatever it wrote stays on dbt-audit so I can inspect it. The next run resets dbt-audit to main again, and those half-written commits stop being reachable from any branch or tag.

Cleanup is Nessie GC, not anything dbt-side. It marks as live every file reachable from any branch or tag within the cutoff (30 days by default), and sweeps the rest. Tags are references too, so a train-football- tag keeps its snapshot's files alive for as long as the tag exists. I give those tags a cutoff of 1 so GC only keeps the commit the tag points at, not 30 days of history behind it.

What decides how long a tag lives is a separate retention job. It keeps a tag if a Model Registry version names it as its training data, if it's under 30 days old, or if it's the newest one. It deletes the rest, and only at the hash it listed (Nessie rejects the delete if the tag moved). It fails closed: if the registry can't be read, it deletes nothing. One caveat: it checks the Kubeflow Model Registry, not MLflow, so an MLflow run that records a tag doesn't protect it on its own.

So yes, I'd use Nessie GC with retention, but put the retention policy on your references (which tags exist), not on files. And run GC in dry-run mode first: mine still runs with --defer-deletes, which only records what it would delete. Its log still says "deleted N files" in that mode, so check the bucket rather than the log line.