QA Automation Engineer & SDET | Founder of aitestingguide.com
QA Automation Engineer & SDET | Founder of aitestingguide.com | Honest reviews of AI testing tools — written by a real tester, not a marketer.
Markdown supported · ⌘↵ to post
Running both against the same local endpoint keeps the comparison honest, and the repo makes it easy to reproduce. The next step I would try is the same test from a separate machine against a deployed endpoint, since localhost hides network latency and connection limits. A staged ramp would also show more than a flat 100 VUs — how p95 degrades as the load builds, not just where it lands.
Comparing the actual test files instead of feature lists makes the trade-offs much clearer. The threshold typo failing loudly is a useful detail — a silent pass would have been far worse in an automated run.
Daniel Collins
The detail that stood out to me was the request count — 3,000 from k6 against 2,874 from JMeter. With a 1-second ramp-up on the JMeter side, that gap is more likely timing than tool performance, which makes the near-identical response times the more honest headline here.
I also liked that the k6 typo fails loudly with exit code 104. A test that silently ignored a broken threshold would be far worse in CI than one that refuses to run. That is a good argument for keeping thresholds in version control and reviewing them like code, rather than treating them as a setting you tweak in a GUI and forget.
For a next run, I would be curious to see the same test with a gradual ramp and a less trivial endpoint, so the tools are stressed on connection handling rather than just returning a tiny JSON payload.