One hundred seeds confirms and converges
what you'll learn · What an N=100 run adds over N=50: tighter means, narrower disagreements between arms, and clearer identification of arms that are statistically identical.
Re-ran the harness at N=100 to refine the 50-seed leaderboard. ts_momentum's lead over baseline holds (+0.132, 82% hit rate). The three three_clock variants converge to exactly +0.798 — what looked like distinct arms at N=10 are statistically identical at N=100. baseline and spread_filter are also indistinguishable. Two findings: ts_momentum is robust; many arms are duplicates.
PR #759 named follow-up #2: “The 100-seed run. N=50 already gives reasonable precision but N=100 would tighten the means by another 1/√2. Tractable in under 2 minutes on the current harness.”
The N=100 run took ~85 seconds and produced 2400 CSV rows. Here’s the top of the leaderboard:
arm mean stdev p25 p75 min max hit%
ts_momentum +0.995 1.095 +0.448 +1.552 -2.358 +3.209 82%
baseline +0.863 1.077 +0.180 +1.588 -1.913 +3.368 79%
spread_filter +0.861 1.076 +0.174 +1.588 -1.913 +3.368 79%
vol_weighted +0.856 1.062 +0.210 +1.545 -1.847 +3.183 78%
blackout +0.851 1.067 +0.162 +1.624 -2.027 +3.151 81%
damping +0.838 1.068 +0.155 +1.625 -2.047 +3.135 81%
three_clock_vol_weighted +0.799 1.210 -0.034 +1.503 -2.337 +3.993 74%
three_clock_portfolio_vol+0.798 1.225 -0.102 +1.549 -2.461 +4.013 72%
three_clock_momentum +0.798 1.232 -0.037 +1.606 -2.479 +4.078 74%
portfolio_vol_gate +0.716 1.139 -0.117 +1.516 -1.562 +3.111 66%
three_clock_vol_regime +0.680 1.237 -0.122 +1.393 -2.004 +3.706 72%
vol_regime_filter +0.649 1.102 -0.181 +1.438 -1.928 +3.222 73%
Finding 1: ts_momentum is robust
N=10: ts_momentum +1.173 vs baseline +0.999 (Δ +0.174)
N=50: ts_momentum +0.910 vs baseline +0.767 (Δ +0.143)
N=100: ts_momentum +0.995 vs baseline +0.863 (Δ +0.132)
The Δ shrinks slightly from N=10 to N=100 but stays robust at +0.13 Sharpe. Hit rate jumps from 76% (N=50) to 82% (N=100) — ts_momentum positively-rewards on more seeds as the seed count grows.
The mean’s stdev should fall as 1/√N. Empirically:
- N=10 stdev: ~1.0 → 1-stdev confidence interval ~0.32
- N=50 stdev: ~1.1 → CI ~0.16
- N=100 stdev: 1.095 → CI ~0.11
The Δ +0.13 is now slightly above the 1-stdev CI of the difference (~0.11). At N=200 the CI would shrink to ~0.08 and the Δ would be 1.6 standard errors above zero. Not formal statistical significance, but consistently directional.
ts_momentum is the harness’s load-bearing winner on single- signal data. The previous “baseline is the high bar” framing from PR #751’s session-summary v2 was wrong.
Finding 2: The three three_clock variants are statistically identical
three_clock_vol_weighted: +0.799 stdev 1.210
three_clock_portfolio_vol: +0.798 stdev 1.225
three_clock_momentum: +0.798 stdev 1.232
Three arms, three different intervention shapes, identical
means at N=100 (within rounding). The differences observed at
N=10 — three_clock_vol_weighted “stacked above” the parents
at +1.058 — were N=10 noise.
This is consistent with the +1.00 correlation found between
three_clock_momentum and three_clock_portfolio_vol in PR
#742’s matrix. They catch the same signal. The
three_clock_vol_weighted variant differs slightly in
mechanism (per-symbol vol normalisation) but ends up at the
same Sharpe.
Implication: the three composite arms can be CONSOLIDATED. Three strategy classes, one effective signal. A future catalog cleanup could pick one as the canonical “three-clock composite” and deprecate the other two — saving operators from N+1 ways to express the same edge.
Finding 3: baseline ≈ spread_filter on this data
baseline: +0.863 stdev 1.077 median +0.972 hit% 79%
spread_filter: +0.861 stdev 1.076 median +0.972 hit% 79%
Identical to 0.002 Sharpe; identical median; identical hit rate; identical min and max. The chop filter is dormant — the top-bottom spread rarely falls below the 4% threshold on this synthetic, so the filter never fires.
PR #742’s pairwise matrix showed baseline vs spread_filter at +1.00 correlation across seeds. The N=100 means confirm: same per-seed sequence → same descriptive stats.
This means: on this synthetic, spread_filter is a no-op. Operators picking between baseline and spread_filter are picking between identical strategies (or the spread_filter is so rarely firing that the small percentage difference is invisible at any seed count).
Finding 4: Multi-factor arms remain the worst
two_factor: -0.666 (worst, hit% 25%)
three_factor: -0.656 (hit% 26%)
four_factor_tuned: -0.614 (hit% 28%)
four_factor: -0.556 (hit% 31%)
Confirms PR #759’s finding. The default mean-reversion-dominant
weights (w_z = -1, w_s = -1, w_k = -0.5) flip the score
direction against the data’s signal direction. Multi-factor
strategies need data with mean-reversion alpha; this synthetic
has momentum alpha.
The four-factor (kurtosis-included) variant is the LEAST-bad of the four — kurtosis penalty partially cancels the mean-reversion tilt. PR #608’s “higher moments add noise faster than signal” rule sharpens: kurtosis is less actively-harmful than skew when the underlying signal is momentum.
The refined ten-rule table
PR #751’s session-summary v2 had 10 discipline rules. The N=100 run adds nothing new but refines two:
| # | Rule (refined for N=100) |
|---|---|
| R1 | Report all four (mean, stdev, min, max) at N ≥ 50 |
| R6 | Mean tightens 1/√N; min/max widen with N; at N=100, ts_momentum’s lead is 1.2 standard errors over baseline |
| R8 | Intervention-point shifts must beat baseline at N ≥ 50 to be load-bearing claims |
The other seven rules are unchanged.
The catalog cleanup question
If three_clock_momentum, three_clock_portfolio_vol, and
three_clock_vol_weighted are statistically identical at N=100,
should the catalog deprecate two of them?
Arguments for deprecation:
- One canonical “three-clock” arm reduces ops surface.
- New operators don’t waste time choosing between equivalents.
- Tests covering 22 strategies are easier than 24.
Arguments against:
- The variants differ in shape (score vs portfolio scale vs vol-normalisation) and might separate on REAL data even though they’re equivalent on synthetic.
- The pairwise correlation is +1.00 ONLY when the gates are dormant; on real data with active vol regimes, the variants would diverge.
- Deprecating shipped strategies sends a stale-thesis signal per ADR-0014.
The honest answer: the catalog should KEEP all three but ADD a note that they’re indistinguishable on this synthetic — the three’s value is differential on real data, not synthetic. The three-clock notes already make this point indirectly; the catalog could state it directly.
The closing observation
The N=100 run did three things:
- Confirmed ts_momentum’s lead.
- Identified three-clock variants as duplicates on this synthetic.
- Identified baseline ≈ spread_filter as duplicates.
None of these were visible at N=10. Two of them (#2, #3) were visible at N=50 but the convergence wasn’t yet tight. At N=100 the picture is clear.
The harness has converged on a stable ranking. The next required step is real data — until then, the synthetic ranking IS the truth we have, and ts_momentum leads.