The parallel holds up surprisingly well. Synchronous multi leader replication and synchronous gradient sync die for the same reason, the slowest participant sets the pace for everyone else. And the fixes rhyme too. Async replication accepts staleness the way async SGD accepts stale gradients, and both just move the hard problem to conflict resolution. This mapping also makes the interview answer much easier to remember than memorizing rules about when to shard. Nice write up.
Branden Floris
The parallel holds up surprisingly well. Synchronous multi leader replication and synchronous gradient sync die for the same reason, the slowest participant sets the pace for everyone else. And the fixes rhyme too. Async replication accepts staleness the way async SGD accepts stale gradients, and both just move the hard problem to conflict resolution. This mapping also makes the interview answer much easier to remember than memorizing rules about when to shard. Nice write up.