A very comprehensive article!
In my experience there are two things that we should be careful of when writing tests to avoid potential false outcomes.
Firstly, we should try to control all variables in the test. If a function relies on the date, and we use something like DateTime.UtcNow, there could be times of the month where this fails but passes at others. In this case, a good idea is creating a separate 'time' service where we can mock such values.
Another is when testing logic that has some aspect of concurrency, as the threading can potentially complete in a different order each time. In this case, it's best to try to get the logic to run single threaded for the tests, if possible.