Property based testing is often introduced by using the implementation of an
add method. The
add method is supposed to add two numbers.
add(1,1) should yield
add(42, 1337) should yield
To test drive the implementation one might start with a simple test case
We’re making the test pass with the simplest implementation
Ok. If all we ever do is adding zeroes the above is enough.
Let’s drive out a smarter implementation
And quickly pass the tests with
Well… guess then we’ll go for
Ah come on.
Let’s stop this nonsense.
That’s usually the point where a property based testing library such as junit-quickcheck is introduced and the tests are rewritten as follows
@Property annotation will make sure the test is run multiple times. 100 times by default. While
@InRange will generate random values inside of the specified boundaries using
Because it is way to hard to have a bad implementation to catch all the random value we’re ultimately forced to give up and write the proper implementation.
We did it. Marvelous. Property based testing rules. Everyone is happy.
Though there are a few issues with the way the concept was taught.
We’re going step through some of them.
Not once has there been a refactoring step. We went through phases in a manner of
Red -> Green -> Red -> Green -> Red [...].
The refactoring step was missing completely.
Additionally the rule As the tests get more specific, the production code gets more generic was not applied.
After seeing the following monstrosity
a refactoring step and a bit of thinking would have led us to
and subsequently help to recognize we might be able to drop the whole
Our standard library will usually offer it out of the box.
and of course Ruby
Ok. Adding might be a stupid example. But so is reversing an array used in the official docs of QuickCheck, the mother of all property based testing libraries.
Again in Haskell, Python and Ruby
As seen above in the test annotated with
The only difference is we have the implementation hidden behind the
All the above might leave the person new to property based testing in a mood of having learned something new, which is great, but having no direct target to apply the gained knowledge to.
Once, after seeing a presentation on the topic, I have asked the speaker in private if he’s using property based testing in his day to day projects. The answer has been no.
As a result you might guess I have an inherent distaste for property based testing.
Nothing could be further from the truth.
In part 2 we will have a look into cases where it does make sense.