Honestly this tracks with my RAG pipeline work. We had a nightmare embedding ingestion script that everyone wanted to torch. Instead we containerized it, added observability (logging + metrics to prometheus), and moved on.
The real win wasn't refactoring. It was making the messy part observable and bounded. Kept it in a separate service, added retry logic, called it done.
Your dbt point is solid too. Data transformation gets weird fast, but at least you can version it and test it. Way better than scattered Python files.
The rewrite itch is real though. Just resist it unless the thing is actually on fire.