Georg Jähnig

Feb 26, 2023

Thanks for this write-up! The article describes well the up- and downsides of browser-based E2E tests on the one side and unit tests on the other side.

However, it would be wrong to assume that these are the only types of automatic tests.

Another one type that comes into my mind are tests that I'd call API-E2E tests: They for example call an API endpoint of the backend (dev or stage server) and check for its response.

Or, in case of a library that is to be tested, they call public (main) methods and check for the return values. Those come probably very close to the advantages of unit tests.

With API-E2E tests, you can have the advantages of both worlds:

these tests are much faster than browser-based E2E tests, as they don't need a browser
there is also not much of additional setup needed, as you have (or should have) anyway a stage or dev server to be set up
these tests are robust even if your inner architecture changes and e.g. you redesign your class and methods' architecture

Especially the latter makes me skeptical about unit tests because they basically force you to double your code: You not only have to write a new function but also a test for a new function.

Or if you want to rename a function, you need to do it at least twice: in both the test and the code (unless your IDE is smart enough to recognize the matching test)

Yes, I do understand that this extra code is meant to be a feature, not a bug, as it automatically gives you a specification and documentation – yet even you seem to agree that this slows down development ("instead of moving fast and break things, I aim to move slow ...").

Further remarks:

I maintain a suite of 3200 unit tests that run in about 30 seconds—only about 1/100 of a second for a test.

True, but when comparing the number of unit tests with the number of E2E tests, we have to remember that 1 E2E tests usually covers a lot of methods, whereas a unit tests usually covers only one method.

Thanks for your response! It's excellent food for thoughts for me on how to argue in favor or unit test.

You are right, "API e2e" can be a perfect tool in numerous instances. My work in 95% on the frontend, and I tend to forget about more backend focused approaches. A few years ago, I was heavily investigating testing JS tools for testing API—as a frontend dev you are doing QA for the backend anyway, so a bit of automatization can help here.

I think a big difference between our projects is that in my case I have in mind:

that at some point I can be outnumbered by junior devs, and I want them to:
- do a bit of their own supervision themselves,
- communicate back the change propositions extremely clearly, so I can review MR efficiently,
I want the project to stay stable, healthy and on a good trajectory even a few years after I will leave it.

For me, those are the reasons to spend extra time writing tests now.

That being said, I'm not sure about the cost of doing TDD. My off hat evaluation is that over the span of a few hours, writing tests slows me at max 25%; buy over a few days benefits compound, and I'm more efficient with unit tests than without them.

One E2E covers a lot, but it makes the error messages less helpful—it tells you some part a long chain of operations is off, and you have to investigate which one it is.

I'm slowly getting to the conclusion that unit test value for solo developers is in unblocking TDD; and the design impact I've talked about in : how-to.dev/how-to-keep-your-units-testable-in-jav…

Search Hashnode