Property-based testing is a software testing technique in which the developer states general rules that must always hold for a piece of code, and a framework checks those rules against hundreds or thousands of automatically generated inputs. In testing, it complements example-based unit tests: instead of asserting that one specific input produces one specific output, it asserts a property, such as “sorting a list never changes its length”, across a wide input space.
When a generated input violates a property, the framework shrinks it to the smallest input that still fails and reports that minimal counterexample. The approach originated with the QuickCheck library for Haskell and is now available in most mainstream programming languages.
How property-based testing finds failures
A property-based test runs a loop of generate, check and shrink. The framework uses generators (also called arbitraries or strategies) to build random values of the required type, calls the code under test, evaluates the property and repeats, typically 100 runs by default in many frameworks.
If any run fails, shrinking begins: the framework tries smaller or simpler variants of the failing input, such as a shorter list, a smaller integer or an empty string, and keeps any variant that still fails. The result is a counterexample that a developer can understand at a glance, rather than a random 300-element list. Most frameworks also print a random seed so that the exact failing run can be reproduced.
| Component | Role | Example |
|---|---|---|
| Generator | Produces random inputs of a given shape | Lists of integers up to 100 items |
| Property | Boolean rule that must hold for every input | Reversing twice returns the original |
| Runner | Executes many trials and collects failures | Runs 100 to 1,000 cases |
| Shrinker | Reduces a failing input to a minimal one | Turns a long list into a two-item list |
| Seed | Makes a random run reproducible | Replays the failing case in CI |
Common patterns for writing properties
The hardest part of property-based testing is finding good properties, because a property must be true for every valid input without simply reimplementing the code. Experienced practitioners rely on a set of recurring property patterns that fit most functions.
| Pattern | Rule it expresses | Typical target |
|---|---|---|
| Round trip | Decoding an encoded value returns the original | Serializers, parsers, compression |
| Invariant | Some quantity never changes | Sorting keeps length and elements |
| Idempotence | Applying an operation twice equals once | Normalization, deduplication |
| Test oracle | Result matches a simpler trusted version | Optimized algorithms |
| Commutativity | Order of independent steps does not matter | Set operations, merges |
| Metamorphic relation | Changing input predictably changes output | Search ranking, numeric code |
A round-trip property is often the best starting point because it is easy to state and catches many encoding bugs. The oracle pattern is especially powerful when a slow but obviously correct implementation exists, for example comparing a hand-optimized search against a linear scan. Metamorphic relations help when the exact correct output is unknown but the relationship between two outputs is known.
Property-based testing compared with other test styles
Property-based testing differs from neighboring techniques mainly in who chooses the inputs and what the assertion describes. In a classic unit test, the developer chooses a few inputs and checks exact outputs; in a property test, the framework chooses the inputs and the assertion describes a general rule.
| Technique | Input source | Assertion | Main strength |
|---|---|---|---|
| Example-based unit test | Chosen by the developer | Exact expected output | Clear documentation of cases |
| Property-based test | Generated by the framework | General rule over all inputs | Finds unanticipated edge cases |
| Fuzzing | Mutated or random bytes, often coverage-guided | No crash or sanitizer error | Finds memory and parser faults |
| Mutation testing | Existing tests on altered code | Tests detect injected faults | Measures test suite quality |
The boundary between property-based testing and fuzzing is blurring, because modern frameworks can use coverage guidance and long-running campaigns. The practical difference remains that property tests run as part of the regular test suite, use typed generators and check semantic rules, while fuzzers usually run for hours and look for crashes. The two styles work best together with example-based tests, which document the known important cases explicitly.
Frameworks and stateful testing
Mature property-based testing frameworks exist for most languages, and they share the same core ideas while differing in shrinking strategy and integration. Choosing one usually follows the project language and test runner.
| Language | Framework | Notable feature |
|---|---|---|
| Haskell | QuickCheck | Original design of the technique |
| Python | Hypothesis | Integrated shrinking and example database |
| JavaScript and TypeScript | fast-check | Works with common test runners |
| Java | jqwik | JUnit 5 engine integration |
| Erlang and Elixir | PropEr and StreamData | Strong stateful testing support |
Beyond pure functions, stateful property-based testing generates random sequences of commands against a system, such as a cache, a queue or a database API, and compares the real system with a simple model after every step. This technique has uncovered subtle concurrency and ordering bugs in production databases and storage systems, because a random sequence of operations explores interleavings that hand-written scenarios rarely cover.
Limitations and common mistakes
Property-based testing is not a proof of correctness: it samples the input space, so rare inputs may never be generated. Its value depends on generator quality, because a generator that rarely produces empty lists, boundary integers or unusual Unicode will miss exactly the cases that break code.
- Restating the implementation as the property, so that the test repeats the same bug.
- Filtering too aggressively with preconditions, which discards most generated inputs and slows runs.
- Ignoring flaky failures instead of recording the seed and adding a regression example.
- Testing slow code with too many runs, which makes the suite impractical in continuous integration.
- Weak properties such as “does not throw”, which pass even when results are wrong.
A good practice is to pin every discovered counterexample as an ordinary unit test, so that regression testing keeps the bug from returning even if random generation never finds it again.
The idea in brief
Normal tests check a few hand-picked cases, like trying a calculator with 2 + 2. Property-based testing instead states a rule that should always be true, for example that adding zero never changes a number, and lets the computer try that rule on thousands of random inputs.
When the computer finds an input that breaks the rule, it keeps simplifying it until it finds the smallest example that still fails. The developer then sees a short, clear case that explains the bug.
A practical scenario
A developer maintains a library that converts configuration objects to a text format and back. The existing suite has 40 example-based unit tests, all passing. She adds one round-trip property: for any generated configuration object, parsing the serialized text must return an object equal to the original.
On the first run, after 63 generated cases, the framework reports a failure and shrinks it to a configuration with a single key whose value is a string containing only a newline character. The serializer had not escaped newlines, so the parser split the value into two lines. The developer fixes the escaping, adds the shrunk input as a permanent unit test and keeps the property running with 500 cases per build. Over the next months, the same property catches two more escaping bugs introduced by new features, each reported as a minimal, readable counterexample.