iSCSI gives you two minutesYou can have enterprise storage, a healthy host and a filesystem doing exactly what it was designed to do, and still lose a node to one network interruption. This is what a firewall racking job did to4h ago · 7 min readjjozzietech.hashnode.dev
The storage failure that was actually an ARP raceThe Windows workload client lost its TrueNAS shares one Thursday morning. Red-X on every drive mapped to that specific server. Mappings to other storage servers on the same VLAN stayed online. Test-NeOct 15 · 15 min readjjozzietech.hashnode.dev
Four lines of patched JavaScript, and the router VM that deleted themTwo small patches to a dependency file called ssdp.js. Four lines of changed JavaScript, total. They kept the fleet's nodes reachable from the internet, and they were the reason every incident on the Oct 1 · 13 min readjjozzietech.hashnode.dev
Five layers, one migration — what a fabric cutover exposed about storageThe plan was a network change with a storage flavour. Move a Windows workload and two TrueNAS servers off the legacy flat network and onto the new dual-TOR fabric on the workload VLAN. Three cutovers.Sep 24 · 16 min readjjozzietech.hashnode.dev
/etc/environment is not read by systemd, and I believed it was for a monthUV_USE_IO_URING=0 was supposed to be the fix. In late August the Flux Cumulus fleet running on Ubuntu 22.04 picked up a nasty failure mode on Node 1: npm install entering uninterruptible sleep, refusiSep 17 · 7 min readjjozzietech.hashnode.dev
