Unit tests are an essential tool in a trustworthy test suite for an Android application or any other software system for that matter. But unit tests themselves doesn’t guarantee that the right features or requirements are tested, even if you did a thorough effort to cover as much code as possible in your entire code base with them. It only proves that the system is actually tested, but says nothing about the quality of the tests. Mutation tests can help with this issue, by measuring the quality of your unit tests by manipulating your code under test. Mutation tests can be seen as the tests of your unit tests.
So, what Exactly is Mutation Testing?
In a nutshell mutation testing is a mechanism to inject different kinds of errors (mutants) into your code base while running your tests. A mutant could be changing a simple conditional in your Java code from
!=. Generally, mutants try to simulate common programming errors like accidentally inverting an if statement, returning
null instead of a real object, etc.
If your test covering the piece of code where this mutant was injected still passes, then the mutant survived. If on the other hand the test fails, then the mutant was killed. As you might already have guessed, the terminology in mutation testing is somehow opposite of normal test results, as killed mutants is a good thing, while surviving mutants is a bad thing, since they indicate, that we didn’t write our test well enough.