Welcome to the Transylvania Java User Group!

Testing randomly

Some times it is useful to throw “random” data at your algorithm to see if it produces the correct result (see fuzz testing). This is especially true if you are writing some fundamental piece of code like a sort algorithm, a scheduler for a thread pool, etc. Then again, knowing that there is a problem is only a little better than living in blissful ignorance, what you ideally would like is a reproducible way to reveal the flaw. For this situation the pseudo-random number generators is your friend. To see an example, look at todays source code.

It does the following:

• It implements a binary search algorithm (no, you shouldn’t do this! this is just as an example! use Arrays.binarySearch)
• For testing purposes:
  1. It generates arrays of variable length (from 0 to 127 in the example)
  2. It fills the array with random values and sorts it (a prerequisite for being able to use binary search)
  3. Checks that for each element the position returned by the binary search is the actual position of the element

The tricky parts of the code are the following:

• Given the same seed, Random will always return the same sequence of numbers (because it is pseudo-random number generator, not a true random number generator – which is perfect for us)
• To add true randomness, we use System.currentTimeMillis() as the seed (ie. the result depends on the time the test was run on)
• To make the failures reproducible, we need to dump the array length and the seed with which Random was initialized. Then, to reproduce it you just call testWithSeed with the given parameters (see testFailures1 for example)
• Random needs to be initialized inside the loop (rather than before it) to make sure that we can easily arrive to the same failure (if we would to initialize it before the loop, we would need to change testFailures1 to this: testWithSeed(0, 5, 1300526812290L);, which would make it more complicated to debug)
• After the code failed for a particular seed, include it in the test suite to make sure that no regression appears later.

In conclusion: sometimes random testing can offer a higher assurance of quality, just make sure that the failure is reproducible.