DRDor Rafael Avrahamofinoptipipe.hashnode.dev·16h ago · 4 min readWhy I stopped guessing at Spark and dbt config valuesI've spent more than a decade building data pipelines, and the part nobody warns you about isn't the pipeline logic. It's the tuning. Executor memory, shuffle partitions, cluster size, thread counts. 00
MNMichael Nocitoinmichaelnocito.hashnode.dev·20h ago · 19 min readOne Migration, Start to FinishNew to this? Read the eight stages first. That page tells you which stage you are in and what to do about it. This page shows all eight happening on one real-shaped job, so it makes more sense second.00
MNMichael Nocitoinmichaelnocito.hashnode.dev·20h ago · 7 min readWhat Data Migration Actually IsData migration is moving a client's records from the system they have to the system they bought. If you are about to run your first one, this tells you what you are signing up for. By the end you will00
MNMichael Nocitoinmichaelnocito.hashnode.dev·20h ago · 4 min readStage 6: User Acceptance TestingUser acceptance testing is where the client's staff open real records in the new system and say whether they are right. It is the last gate before anything becomes permanent, and it fails for one reas00
MNMichael Nocitoinmichaelnocito.hashnode.dev·21h ago · 6 min readScope Creep and the Project PlanScope creep is not one big request, it is fifteen small reasonable ones. The answer is not refusal. It is showing what each request costs on the plan, and letting the client decide with that in front 00
MNMichael Nocitoinmichaelnocito.hashnode.dev·21h ago · 6 min readWhen the Client Goes QuietA client who stops replying is the most common reason a migration stalls, and the response most people reach for is a polite nudge that changes nothing. This article gives you an escalation ladder tha00
MNMichael Nocitoinmichaelnocito.hashnode.dev·21h ago · 5 min readAsk the Right Questions FirstMost migration trouble traces back to a question nobody asked in week one. This is the list I run through before agreeing to a date, grouped by what each answer protects you from. The short version. F00
MNMichael Nocitoinmichaelnocito.hashnode.dev·21h ago · 6 min readStage 2: Profile the SourceProfiling is looking at the real data to find out what is actually in it. It takes a day or two and it is the only stage whose output tells you whether your timeline is honest. This article gives you 00
MNMichael Nocitoinmichaelnocito.hashnode.dev·21h ago · 5 min readWhat Goes Wrong, and What It CostsEvery one of these has a cheap check that catches it early and an expensive version that catches it late. This is the list, ordered by how much the late version costs. The short version. Almost all mi00
MNMichael Nocitoinmichaelnocito.hashnode.dev·21h ago · 6 min readStage 1: Kickoff and ScopeA kickoff is not a get to know you call. It is where you define the project, and anything you fail to settle here you will renegotiate later at a worse moment. This article gives you the agenda, the f00
