We've our Rest API server built on NodeJS and we've started to implement "test".
Our app consists of a lot of DB calls (mostly using ORM), API calls to FB and other 3rd party services, Message queue etc.
We started implementing Unit tests. However, it looks like we need to mock 80% of the code and found it too much time-consuming. Moreover, at last, we will not be 100% confident with our code
A good example would be a createUser function written in raw SQL. It worked perfectly if the name is "Gijo". But will fail if the name is "O'bama". If I write mocks for createUser function, both Gijo and O'bama will succeed in the unit test but will fail in production (correct me if I'm wrong).
Another way I found is to write integration tests instead of unit tests. This seems to be much more efficient and gives us more confidence. And it's easy to test, sending multiple post requests to localhost/users with different names like "Gijo", "O'bama" etc and check whether it returned 200 and it's correctly inserted to our DB. In such a way we're also making sure our Queue, API calls, DB and everything is working.
What will you suggest in this scenario? Any advice?
I write both unit tests and integration tests. My tests are about 75% of the code base, but finding (and fixing) bugs is super easy this way.
Unit tests test functions as code, not as functionality. I try to write tests for all possible branches of code. This may sound cumbersome, but most of the tests are boilerplate, for what i have code snippets in my editor. As there are a lot of functions that modify the database, i always run unit tests in a separate environment and/or with the ORM being mocked. These are important as if i need to change tests due to code change, it makes me take a good look if all callers of the changed function is updated.
Integration tests are harder to write. I usually use BDD (Behaviour Driven Development) with a natural-like language, so even non-tech colleagues can write tests. A lot of these tests are destructive so they run in an isolated environment, but in an infrastructure identical to the production one.
As a result i barely introduce bugs in the code (the business model is a different story, though). As a con, it takes time to create the necessary environments, but i wouldn’t daresay it doesn’t worth it.
I have not written any test. But I see most of the developers write test for each single task. This is what I feel very bad. Because, task requirement may change in the middle. You'll need to write the test again and again.
So, I would like to suggest:
We have stopped writing unit tests for any code that has side effects.
This has an interesting side-effect (pun-intended) that because integration-tests take longer to write and run people tend to write more code that is side-effect free and isolate side-effects to interaction boundaries.
Well maybe one just per default escapes the data as it should be.
This sounds like integration test case since you guys are testing the rest API and usually an API is only to be seen as what it returns not how it behaves internally. You test against the interface in other words .... your REST-API should not care if something is in the database. it should only be concerned with the defined behaviour of the REST-API.
So for this intent I would mock the BL below because it's a response behaviour and you can purposly force all states it should represent without the overhead of actually doing anything of great calculation costs below. Unless you did not loosely couple then you should use an in memory SQL-Lite database with fixtures to mock, so at least the overhead is less and you can throw away the database after testing.
if you want to test your business-logic you have to test it as a functional test which is to ensure it works as intended. this is where you actually should compare the state of your application instead of the communication with your application.
Doing unitests afterwards is a pain in the ass esp since doing unitests from the beginning helps you to create better software design. That's to me the main purpose of it, the other things like guarantees are nice, but tests just only ensure quality in the metric around stability not the others.
Maybe just start with only blackboxing the API since this is your current task and afterwards go in for the business logic. Actually you should've done it the other way around but you have to start somewhere and this is an achievable goal.
at least in my opinion
I'll highlight a conceptual information on unit tests first --- skip to last paragraph for the answer
Unit tests are written to confirm the behaviour of an atomic logical task within its service boundary. Any external supporting systems that comes outside the boundary is usally mocked. Hence we mock the database, third party IO services because they're external systems.
The above statement doesn't mean we should stay cool because all unit tests worked. It's like saying my car engine, wheels, steering and headlights work fine independently so I'm sure car should run. No. You need to "assemble" them and then run the assembled car to know if it actually runs. You need to wire unit tests together along with necessary external systems which is what people call Integration Tests.
Now enough of concepts.
Coming to one of your usecases creating a user; you may call it as a logic task but I would call it as persisting in an "external" datastore so I wouldn't write a unit test for that case unless there is a logic involved along with db persist, instead I would an integration test without mocking database. Sometimes there will be a validation logic involved like you said "O'bama" for which I would fit it in integration test otherwise it would be an overkill to write unit test only to test validation logic with mocked DB.
This is how I follow the testing approach and there can be better approaches too.