Fuzzing and automated testing

In short: In fuzzing you automatically feed a program huge amounts of random or semi-random input to find crashes and security holes.

In more detail: A fuzzer generates inputs, starts the target and observes. Coverage-guided fuzzers (AFL++, libFuzzer) notice which inputs reach new code paths and keep mutating exactly those. With sanitizers (AddressSanitizer) silent memory errors show up too.

In Depth

• Kinds: Blind (random), mutation-based, grammar-based, coverage-guided.
• Sanitizers: ASan (memory errors), UBSan (undefined behaviour), TSan (data races).
• Harness: Small function that passes fuzz data to the function under test.
• Property-based testing: Properties instead of examples (for example decode(encode(x)) == x), with libraries such as Hypothesis, fast-check.
• Corpus: Collection of interesting inputs, maintained along the way.
• CI fuzzing: OSS-Fuzz, ClusterFuzz.
• Finds: Buffer overflows, crashes, infinite loops, parser errors.

See also: OWASP Top 10, Lexer, parser and syntax tree