Standard Transformer Attention vs. Attention-Residuals: A Practical Comparison

If you've been keeping an eye on GitHub trending lately, you've probably noticed MoonshotAI/Attention-Residuals climbing the charts. It's one of those repos that makes you stop and think — "wait, we've been doing residual connections in transformers ...