What 27 arms told us — session research log v3
what you'll learn · Eleven load-bearing claims this session's harness work produced — eight from v2, three added since — and the discipline rules each supports.
The v2 summary at 24 arms / 8 claims is now stale. The v3 covers 27 arms / 11 claims, including the small-N discipline corpus (two findings disconfirmed at higher N), the pairwise rule's four iterations and final form, and the dual-signal stacking confirmation. This note replaces v2 as the top-of-stack index for the session's research arc.
The original session-summary
(what-19-arms-told-us) named four
claims at PR #689. The v2
(what-24-arms-told-us) added four
more at PR #751. Since v2, the harness grew from 24 to 27 arms;
the strategy catalog from 22 to 25 strategies. Three new claims
emerged, two earlier claims were refined or disconfirmed.
This is the v3 index.
Status of the v2 claims
| # | Claim | v2 status | v3 status |
|---|---|---|---|
| 1 | Strategy shape > factor count | established | reinforced — entry-rule axis confirmed leader at N=100 |
| 2 | Composite-horizon beats single-window | established | refined — three three_clock variants are equivalent at N=100 |
| 3 | Mean Sharpe rankings hide variance | established | reinforced — min Sharpe widens with N, mean tightens |
| 4 | Composite stack-vs-interfere data-dependent | established | confirmed empirically at N=100 dual-signal |
| 5 | Per-symbol vol intervention is wrong | established | partially-disconfirmed — clustered-vol gap was N=3 noise |
| 6 | Cross-symbol recovers Sharpe | established | refined — recovery shrinks at N=100; still beats per-symbol variants |
| 7 | Same-stage composition stacks | established | confirmed; iteration 3 of pairwise rule |
| 8 | Pairwise correlation predicts stack | established | iterated to final form: 4 conditions, 5/5 direction predictions |
Claim 9: At N=10, +0.1 Sharpe deltas don’t survive at N=50+
Both small-N “wins” this session collapsed at higher seed counts:
vol_regime_filter << baselineon clustered vol (PR #717, N=3): claimed Δ −1.3. At N=30: Δ +0.002.three_clock_vol_weighted > baselineon single-signal (PR #746, N=10): claimed Δ +0.10. At N=50: Δ −0.002.
The pattern is the same as the original PR #753 finding about the latter: small-N variance dominates Δs below ±0.5 on this harness. Establishes the regression-test threshold (N≥30 for research-note claims; N≥50 for regression tests).
→ Documented:
the-clustered-vol-finding-was-also-small-n +
fifty-seeds-reveal-the-tie.
Claim 10: The pairwise rule has four iterations
Each iteration of the pairwise-correlation rule was triggered by a composite result that confounded the previous iteration:
- PR #710: Low pairwise correlation predicts stack.
- PR #742: Compute the WOULD-BE composite’s correlation.
- PR #748: Same-stage required; mixed-stage interferes.
- PR #770: Composition mechanism must PRESERVE both parents’ decision rules (no sign-vs-rank conflict).
Final form: four conditions all required for stacking. Empirical accuracy across 5 composites tested:
- 5/5 direction (stack/interfere/no-conflict predictions)
- 0/5 magnitude on single-signal (no composite cleanly stacked at N=100)
- 1/1 magnitude on dual-signal (three_clock_vol_weighted stacks at N=100 with –mean-revert-bps 100)
→ Documented:
the-pairwise-rule-final-form +
dual-signal-n100-composites-stack.
Claim 11: ts_momentum is the harness leader on directional data
At N=100 on default single-signal synthetic:
ts_momentum: +0.995 (hit rate 82%)baseline: +0.863 (hit rate 79%)
The entry-rule discipline (|return| > threshold) closes time-series chop the cross-sectional ranking can’t filter. The lead is 1.2 standard errors above zero; pinned as a regression test in PR #782.
→ Documented:
why-ts-momentum-leads-at-n100 +
hundred-seeds-confirms-and-converges.
The combined discipline rules (v3)
| # | Rule | Source |
|---|---|---|
| R1 | Report all four (mean, stdev, min, max) at N ≥ 50 | ADR-0060 |
| R2 | Equal-leverage controls for Sharpe comparisons | ADR-0060 |
| R3 | Run composite arm before declaring two top arms | ADR-0060 |
| R4 | Entry rule does more work than score rule | Claim 1 + 11 |
| R5 | Multi-window single-factor beats single-window | Claim 2 |
| R6 | Min Sharpe widens with N; N=50+ estimates more honest | Claim 3 |
| R7 | Composite stack-vs-interfere data-dependent | Claim 4 |
| R8 | When N shapes fail, change intervention point | Claim 5 + 6 |
| R9 | Same-stage stacks; mixed-stage interferes (when gate dormant) | Claim 7 |
| R10 | Pairwise correlation + same-stage + sign-preservation predict stack | Claim 8 + 10 |
| R11 | Small-N (under 20) Δs below 0.5 are noise | Claim 9 |
The session’s research arc, in one paragraph
The session opened with PR #689’s four-claim summary at 19 arms
and 17 strategies. By PR #751, eight claims at 24 arms / 22
strategies — with vol-intervention and composite stacking as
two emerging research threads. By PR #777, eleven claims at 27
arms / 25 strategies, with the small-N discipline corpus
(Claim 9) acknowledging that two earlier claims (PR #717 and
PR #746) didn’t survive higher seed counts. The pairwise rule
iterated four times. Two regression tests (PR #782 + PR #787)
pin load-bearing findings against future drift. The catalog
documents three equivalence classes
(hundred-seeds-confirms-and-converges)
and both success and failure modes of composition.
What the next session inherits
A working platform:
- 27-arm A/B harness with multi-mode synthetic data (i.i.d., dual-signal, vol-clustering)
- 25 registered strategies covering the entry-rule × score-rule × event-aware × composite design grid
- Analysis script with percentile + pairwise-correlation diagnostics
- Two CI-enforced regression tests
- ADR-0060 measurement protocol
- 11 discipline rules with documented evidence
A documented research arc:
- 40+ research notes traceable from claim to evidence to refinement
- Pairwise rule’s complete evolution from one-line to four- condition final form
- Both positive findings and disconfirmed claims
A clear next step:
- Real-data run. All claims here are synthetic-specific. Real US equities + Polygon bars + the live FOMC calendar is the next test surface. The harness, the catalog, and the discipline rules are the platform that makes the next experiment cheap to attempt.
Closing observation
The corpus has accumulated faster than the rules — 40+ research notes for 11 discipline rules. The notes capture the territory; the rules capture the path through it. Both are needed: rules without notes lose the empirical grounding; notes without rules lose the navigation.
The harness, the analysis script, the regression tests, and the ADRs are the artifacts that persist. The next operator picks them up where this session leaves them.