Fifty seeds, twenty-four arms — the full leaderboard

PR #753’s 50-seed run focused on the four-arm comparison around three_clock_vol_weighted. The CSV contains all 24 arms; running scripts/analyse_harness_csv.py /tmp/h50.csv --top 24 produces the full leaderboard:

arm                        mean    stdev    p25     p75     min      max    hit%
ts_momentum              +0.910   1.117  +0.246  +1.636  -2.036  +2.984    76%
vol_weighted             +0.795   1.022  +0.006  +1.486  -1.030  +2.871    74%
spread_filter            +0.768   1.057  -0.076  +1.572  -1.091  +2.879    70%
baseline                 +0.767   1.065  -0.096  +1.572  -1.091  +2.879    70%
three_clock_vol_weighted +0.765   1.149  -0.008  +1.386  -2.337  +3.391    74%
blackout                 +0.745   1.070  -0.078  +1.495  -1.108  +2.845    74%
three_clock_portfolio_vol+0.732   1.207  -0.093  +1.459  -2.461  +3.301    72%
damping                  +0.727   1.066  -0.065  +1.478  -1.109  +2.820    74%
three_clock_momentum     +0.726   1.215  -0.142  +1.576  -2.479  +3.290    74%
portfolio_vol_gate       +0.707   1.175  -0.154  +1.563  -1.562  +3.111    64%
three_clock_vol_regime   +0.619   1.199  -0.116  +1.357  -1.869  +3.706    72%
vol_regime_filter        +0.573   1.180  -0.572  +1.507  -1.513  +3.222    66%
long_only                +0.510   1.111  -0.399  +1.256  -1.947  +2.604    70%
equal_risk_long_only     +0.510   1.111  -0.399  +1.256  -1.947  +2.604    70%
spread_filter_tuned      +0.356   1.143  -0.378  +1.065  -2.372  +3.223    58%
vol_transition_filter    +0.288   1.202  -0.412  +1.028  -2.361  +2.477    58%
drift                    +0.256   1.003  -0.430  +0.892  -1.777  +2.356    64%
vol_penalty              +0.083   1.155  -0.611  +0.839  -2.811  +2.287    54%
reversal                 -0.256   1.003  -0.892  +0.430  -2.356  +1.777    36%
four_factor              -0.466   1.118  -1.292  +0.372  -3.015  +1.628    36%
mean_revert              -0.490   1.156  -1.103  +0.055  -2.901  +1.872    28%
three_factor             -0.548   0.990  -1.235  +0.163  -2.645  +1.357    32%
four_factor_tuned        -0.564   1.032  -1.347  +0.155  -3.024  +1.354    28%
two_factor               -0.653   1.049  -1.350  +0.022  -2.798  +1.442    26%

Three findings worth naming.

Surprise 1: ts_momentum is the leader, not baseline

The session-summary v2 named baseline as the high-bar for all vol intervention experiments. The 50-seed run shows ts_momentum is materially higher: +0.910 vs baseline’s +0.767 — a +0.143 Δ, larger than the 1-stdev noise floor (1.117).

The hit_rate column tells the same story: ts_momentum at 76% vs baseline at 70%. ts_momentum positively rewards on more seeds.

This is consistent with PR #557’s “strategy shape beats factor count” rule — ts_momentum uses entry-rule discipline (only trade when |return| > threshold), which closes time-series chop that the cross-sectional ranking can’t.

The previous session-summary v2 (PR #751) needed an updated “top arm” claim. The 50-seed leaderboard makes it: ts_momentum.

Surprise 2: Multi-factor strategies are all NEGATIVE

two_factor:        -0.653   ← worst arm in the catalog
four_factor_tuned: -0.564
three_factor:      -0.548
four_factor:       -0.466

All four multi-factor variants underperform baseline by ~0.5-1.4 Sharpe. The “more factors = better signal” intuition is rejected on this synthetic — adding zscore, skew, kurtosis to momentum DEGRADES the score.

The mechanism: the default factor weights flip the strategy from momentum-dominant to mean-reversion-dominant (w_z = -1, w_s = -1, w_k = -0.5). On single-signal FOMC-drift data, the underlying signal IS momentum; mean-reversion-dominant scores trade against it.

The hit rates confirm: 26-36% (vs baseline’s 70%) — these arms LOSE on most seeds. This is a robust negative result across all four variants.

Discipline implication: PR #557’s rule sharpens to “more factors = more noise unless the factor weights match the data’s signal direction.” Default weights (mean-reversion-dominant) only work when the data is mean-reversion-dominant.

Surprise 3: portfolio_vol_gate’s value isn’t in the mean

portfolio_vol_gate:  mean +0.707  stdev 1.175  hit% 64%
baseline:            mean +0.767  stdev 1.065  hit% 70%

The cross-symbol intervention loses to baseline on every column. Δ mean −0.060, Δ stdev +0.110 (wider, not narrower), hit_rate −6 percentage points.

The 10-seed result (PR #732) showed portfolio_vol_gate’s stdev LOWER than baseline (+0.610 vs +0.677). At N=50, it’s HIGHER (+1.175 vs +1.065). The smaller seed count caught a lucky stdev; the larger seed count tightens the estimate to a worse value.

PR #737 claimed portfolio_vol_gate recovered 75-80% of the Sharpe gap that per-symbol variants sacrificed. At N=50, the recovery is closer to 80% — but the base (baseline) is the same as before, so the absolute gap is comparable to the per-symbol variants now that they’re all under more measurement.

The honest reading: portfolio_vol_gate’s mean and stdev advantages don’t persist at higher seed counts. Its claim to fame is that the worst-case (min Sharpe −1.562) is moderately better than baseline’s (−1.091)… wait, no, baseline’s min is BETTER. portfolio_vol_gate is worse on min too.

The cross-symbol intervention point WAS the right framing relative to per-symbol filters — but it doesn’t actually beat baseline at N=50 on any metric. The “intervention-point rule confirmed” claim from PR #737 needs softening.

What the leaderboard rules out

• Not “ts_momentum is the answer.” +0.910 vs +0.767 is significant on this synthetic but might not generalize. Real markets have multiple alpha sources; ts_momentum’s entry-threshold discipline targets one (time-series momentum).

• Not “multi-factor is doomed.” The default weights happen to mismatch this synthetic’s signal direction. Tuned weights (the four_factor_tuned arm) might work on different data; here both default and tuned variants underperform.

• Not “portfolio_vol_gate is worthless.” It still has the property of “smoother ride” on the OPERATIONAL metric (lower per-bar drawdown during high-vol windows) even if the per-seed Sharpe doesn’t improve. An operator with leverage constraints might prefer it.

The refined discipline rules

PR #751’s session-summary v2 had ten discipline rules. The 50-seed full leaderboard refines two:

• R6 (originally: “Higher mean ≠ better choice; min Sharpe is the gate”) — adds: min Sharpe widens with seed count; N=50 estimates are more honest than N=10 estimates.

• R8 (originally: “When N shapes fail, change intervention point”) — adds: the new intervention point must beat baseline at N=50, not just at N=10. Lucky small-N results don’t survive higher seed counts.

The next experiments

1. ts_momentum + entry-rule analysis. ts_momentum wins at N=50; understanding which seed conditions it fails on (24% of seeds had negative Sharpe) would refine the strategy’s deployment criteria.

2. The 100-seed run. N=50 already gives reasonable precision but N=100 would tighten the means by another 1/√2. Tractable in under 2 minutes on the current harness.

3. Real-data run. Still required. Synthetic results + stable ranks at N=50 are the platform; the next experiment is pointing this at SP500 + Polygon bars.

The synthesis

At N=10, the harness output is suggestive. At N=50, it’s diagnostic. The previous-summary’s “first composite to beat baseline” claim at N=10 is, at N=50, a TIE (and ranks 5th out of 24 by mean). The “intervention-point rule confirmed” at N=10 softens at N=50.

These aren’t failures; they’re the discipline working. Each small-N claim is provisional; the higher-N run is what we should trust.

The harness has been doing its job all along. The new note, informed by a higher seed count, is just a tighter view of what was always there.