A/B Testing at Scale: Power, Sequential Testing, and Practical Pitfalls

What peeking, low power and too many metrics do to your error rates, measured with 20,000 simulated experiments.