Mutation testing

In short: In mutation testing, a tool deliberately changes the program code slightly (for example > to >=) and checks whether the tests notice.

In more detail: High test coverage only says which lines ran, not whether the tests catch bugs. Each change is called a “mutant”. If a test fails, the mutant is “killed”; otherwise it “survives” and reveals a test gap.

In Depth

• Mutation score: Share of killed mutants. The higher, the more meaningful the tests.
• Typical mutations: Swapping operators, changing constants, negating conditions, removing lines.
• Cost: Computationally heavy because tests run many times. Often only for critical modules.
• Equivalent mutants: Some changes do not alter behavior and cannot be killed.
• Tools: Stryker (JavaScript/C#/Scala), PIT (Java), mutmut (Python).

See also: Property-based testing