Property tests catch cross-strategy bugs that per-strategy tests miss

The platform has three event-aware strategy wrappers (XsMomentumPostFomcDriftStrategy, XsMomentumPostEventReversalStrategy, XsMomentumWithEventDampingStrategy) plus the blackout. Each shipped with its own dedicated tests, each was code-reviewed, each declared the same citation contract:

Empty weights → empty cited_event_artifacts. (ADR-0058)

Two of them honored it. The third — damping — silently set cited_event_artifacts to a real event ID and then returned {}. No per-strategy test caught the bug.

The discovery happened only when a single parametrized property test was added — one assertion, three strategies, one fail:

@pytest.mark.parametrize(
    "strategy_factory",
    [drift_factory, reversal_factory, damping_factory],
    ids=["drift", "reversal", "damping"],
)
def test_cold_inside_window_clears_citation(self, strategy_factory):
    s = strategy_factory(self._fired_table())
    weights = s.compute_weights({"X": 100.0, "Y": 100.0})
    assert weights == {}
    assert s.cited_event_artifacts == ()

The damping branch failed. The bug had been live since the strategy shipped.

Why per-strategy tests aren’t enough

Each event-aware wrapper had test coverage of:

• Cold start (history not warmed → empty weights).
• Citation set when inside the gating window.
• Citation cleared when outside the gating window.

But none of them combined the two: inside the window AND cold inner. That’s the corner where ADR-0058 lives.

Per-strategy tests are written from inside the strategy’s mental model. The author of XsMomentumPostFomcDriftStrategy thought about drift, wrote tests for drift, fixed the bug for drift (prompted by /code-review). The author of the damping strategy thought about damping, wrote tests for damping, missed the same corner. The bug is a family-level invariant violation, not a per-strategy logic bug.

What changed when the property test landed

The test surface for ADR-0058 is no longer “we hope each strategy’s author remembered.” It’s:

Adding a new event-aware wrapper to tests/test_strategies.py::TestADR0058CitationContract’s parametrize list is part of shipping the wrapper.

That’s a closed-world rule — the parametrize list is grown by hand, not auto-discovered. The cost is one line per new wrapper. The payoff is a regression test that doesn’t depend on the new wrapper’s author rediscovering the contract.

ADR-0058’s “Test contract” section was added the same day as the property test, with the parametrize convention spelled out so the next person hitting this code knows what to update.

When property-based contracts beat per-instance tests

Three conditions to look for:

1. A contract that crosses an API boundary. Citation contracts live in the wrapper’s compute_weights output, not in any single internal field. The output is the only thing the runner sees — and the only thing the contract can be expressed against.

2. A finite set of consumers. “Every event-aware wrapper” is a closed set the contributor must extend by hand. Three today, maybe five next year, never infinite. A property test that knows the set is checkable; an open-world property test becomes hypothesis-shaped (Hypothesis-the-library) which is a different cost tier.

3. A bug that’s invisible to per-instance testing. If every wrapper’s per-strategy tests cover the contract corner, the property test is redundant. If — like ADR-0058 — the contract corner is the conjunction of two states that no one’s per-strategy test combined, the property test is the only place it lives.

The platform’s discipline: when shipping a new event-aware wrapper, add a row to the property-test parametrize list AND update the existing per-strategy test file. The two together catch the bugs neither alone catches.

What this isn’t

• Not a generic argument for property-based testing everywhere. Property tests have a real cost: they parametrize over types the author needs to keep in mind, and they fail in less-specific ways than per-instance tests (the failure says “drift was fine, damping wasn’t” rather than “damping’s line 1273 did X”). The per-strategy tests stay valuable; the property test adds a layer.

• Not a claim that this would have been caught by formal methods. The bug is a missed branch in one method; no type system or contract framework would have caught it without the same author-mental-model gap. The fix is the parametrize list, not Liquid Haskell.

• Not a retro-claim about /code-review. The /code-review on the damping PR did flag related issues (release_id validation, boundary semantics). It didn’t flag the cold-inner citation branch because the contract wasn’t yet codified — ADR-0058 landed after the damping strategy. The fix is the codified contract + the property test that enforces it, not a stronger reviewer.

The discipline rule

When a contract applies to “every consumer of pattern X” and X has more than two concrete instances, write a property test over the family. Per-instance tests stay; the property test stacks on top. The parametrize list is the contract’s enforcement surface.