On my current project we're building an asynchronous message based system with various outputs, in particular a file for consumption by a mainframe.
On a number of occasions tests have been written that assert that something has not happened.
When you're testing synchronous processes this is a perfectly reasonable assertion to make, when the process returns all the work will be done.
For asynchronous processes?When the process returns then you have no direct way of knowing if it has completed.
For many people the most obvious way of checking that something has not happened in an asynchronous process is the counterpart for checking that something has happened asynchronously. You put in a timeout.
The problem with this is the likelihood of a given outcome. When expecting something to happen asynchronously you only have to wait until it has happened which means that you only ever wait until the timeout when something has gone wrong. When expecting something not to happen, you end up waiting for the time out most of the time. This means that for every assertion that nothing will happen you end up adding a fixed timeout that causes your test phase to increase in duration.
When you use a time out to wait for something not to happen, you risk setting the timeout too short and the test becoming invalid because the thing you asserted wouldn't happen occurs after the test completes. This can lead to further increases in the timeouts to ensure that the tests remain valid.
There is a related problem with testing asynchronous processes where the test only checks an intervening step in the process before moving on the next test. This leaves you open to side effects from the processes started by the preceding tests causing hard to diagnose problems with your later tests. This is particularly likely with tests that assert that something doesn't happen as they have a greater chance of leaving the test before the process under test has completed.
How do we avoid these problems and still be able to make assertions that something hasn't happened?
The 'Sentinel' pattern is a good place to start.
Send a second request after the first that will produce a result after the first has completed, in this way you can assert that if you only see the result of the second request then you can state that the first request hasn't produced anything. Maybe you can ask the asynchronous process to do two things in one request and can then assert that only the effects of the additional work are visible.
Alternatively you can look at other ways of knowing that the process under test has completed. Check the logging for a statement that is only produced when the process has completed or maybe introduce an event or message that is produced when the process completes regardless of whether the other output is produced.
It is always a bad idea to expect 'nothing' to happen asynchronously, always expect something.
