Unit testing nowadays is pretty much always done using a technique called example-based testing. The method is simple; we run a series of examples against an algorithm, and we validate that we get the expected result.
e.g.
[Fact]
public void GivenThree_WhenCalculate_ThenFiveIsReturned()
{
// Given
var input = 3;
// When
var result = Calculator.Calculate(input);
// Then
Assert.Equal(5, result);
}
This technique is relatively easy to understand and implement. However, it's far from being all rainbows and unicorns. In this article, we'll go over property-based testing, an alternative approach that shines where example-based testing tends to fall short.
The main problems with those tests are that:
- They don't scale
- They don't help you find edge cases
- They are brittle
- They don't explain the requirement
- They tend to be too specific to the implementation
We all do this
If you are like most of us, mortals, you are probably terrible at using TDD, which means that in the best-case scenario, you add your tests after coding the actual method. Moreover, I bet that you write an empty test shell calling your method under test, take the output, and then write your assertion with that value. Let's be honest. The temptation to reach a better code coverage is just too strong that we have all done it at some point. Then, if you have a bug in your code, you effectively only coded a buggy test.
Property-based testing
Property-based testing is an alternative approach that can deal with all the shortcomings of example-based tests. However, it's by no means a full replacement, as example-based tests are still excellent in many cases. The best approach is to have the right mix of the two in your test suite.
What's a property
When we are talking about properties here, we don't mean properties of C# classes. We use the term property here in a more abstract way. Think of it as a characteristic, trait, attribute, feature, or quality of an algorithm.
Simple definition:
A property is a condition that will always be true for a set of valid inputs.
Testing method
The first thing to do when you want to create a property-based test is to understand and represent the relationship between inputs and outputs. Then, use randomly generated inputs to express deterministic expected results.
Said like that, it seems complex, but you'll see, it's more straightforward than it sounds.
Example
Let's try the technique with a simple example; the Add method. The signature looks like this.
int Add(int number1, int number2);
If you try to test this method with examples, you'll never be done for sure, as there's an infinite combination of numbers that you can add together. So how can we be sure that our Add method is behaving correctly in every situation?
First, try to think of what makes the addition different from the other mathematical operations like subtraction and multiplication.
Properties
The first property is called commutativity. For those of you that don't have a math major, here is a simple definition:
When I add two numbers, the result should not depend on the order of the parameters.
2+3 = 3+2 // True
2-3 != 3-2 // False
2*3 = 3*2 // True
This property is true for addition and multiplication, but not for subtraction. Translated to a test in C#:
// When
var res1 = Add(input1, input2);
var res2 = Add(input2, input1);
// Then
Assert.Equal(res1, res2);
The second property is called associativity.
When I add two numbers, the order of the operations doesn't matter.
(2+3)+4 = 2+(3+4) // True
(2-3)-4 != 2-(3-4) // False
(2*3)*4 = 2*(3*4) // True
Again, this property is true for addition and multiplication, but not for subtraction. Translated to a test in C#:
// When
var res1 = Add(Add(input1, input2), input3);
var res2 = Add(input1, Add(input2, input3));
// Then
Assert.Equal(res1, res2);
The third and last property of the addition that makes it unique is called identity.
Adding zero is the same as doing nothing.
2+0 = 2 // True
2-0 = 2 // True
2*0 != 2 // False
At last! We found something to differentiate the addition and the multiplication. Translated to a test in C#:
Assert.Equal(Add(input, 0), input);
Conclusion
With three simple tests, we've been able to cover an infinite number of examples. With these, it's impossible to write an implementation of the Add method that doesn't behave properly. That's precisely the way mathematicians define concepts. They describe the properties. If you look up the definition of the addition, you'll find something interesting. It's defined as commutativity, associativity, and identity.