Skip to main content
2026-05-22 · 4 min read · ← 3 · research · harness · composition

Fifty seeds reveal the tie

what you'll learn · Why the first composite-arm result to beat baseline at N=10 reverted to a tie at N=50, and what this confirms about the discipline rule of trusting only multi-seed-survivable claims.

At N=10, three_clock_vol_weighted beat baseline by +0.10 Sharpe — directionally encouraging but within the stdev (1.22). At N=50, the composite ties baseline at +0.765 — within rounding of baseline's +0.767. The directional signal didn't survive the higher seed count, exactly as the session-summary v2 note's caveat predicted.

PR #746’s headline result:

                                 mean    stdev   max
baseline:                       +0.957   1.335  +2.879
three_clock_vol_weighted:       +1.058   1.216  +3.347  ← above

The note (the-pairwise-rule-predicted-this-one) called this out:

+0.101 vs baseline is well below stdev 1.216. Statistically the composite ties baseline at this seed count. The direction (above, not below) is the load-bearing observation; the magnitude needs more seeds.

PR #751’s session-summary v2 named the follow-up:

Higher seed count. 10 seeds isn’t enough to claim three_clock_vol_weighted beats baseline statistically. Re-run at N=50 or N=100.

This note documents the re-run.

The 50-seed result

arm                       mean    stdev    min      max
vol_weighted             +0.795   1.022   -1.030   +2.871
baseline                 +0.767   1.065   -1.091   +2.879
three_clock_vol_weighted +0.765   1.149   -2.337   +3.391
three_clock_momentum     +0.726   1.215   -2.479   +3.290

The Δ between three_clock_vol_weighted and baseline: −0.002. Within rounding. The 10-seed +0.10 reverted to a tie.

vol_weighted (one of the composite’s parents) actually shows the highest mean at N=50 (+0.795). At N=10 it was +0.930 — also not a meaningful signal. The seed-by-seed variance dominates the small differences between arms.

Two more findings the 50-seed run surfaces

The min Sharpe is much wider

N=10:  three_clock_vol_weighted min -0.932
N=50:  three_clock_vol_weighted min -2.337

The min Sharpe more than doubled. Higher seed count → more chances to draw a bad seed → wider observed worst-case. The N=10 min was a lucky observation; the N=50 min is closer to the true worst-case for the data distribution.

This sharpens ADR-0060 Rule 1 (rank by operator’s metric): operators using min Sharpe need MORE seeds, not fewer. The mean tightens slowly with seed count (1/√N); the min/max ENLARGES with seed count (because more draws = more extremes). A drawdown-constrained operator running at N=5 is reading a suspiciously-good worst-case.

The arm ordering shuffled

Arm N=10 rank N=50 rank
baseline 1 2
vol_weighted 2 1
three_clock_momentum 3 4
three_clock_vol_weighted 4 3

The “first composite to beat baseline” became the third-ranked arm at higher seed count. The rank flips are all within stdev, but they show how shaky a 10-seed ranking is when the true deltas are small.

What this confirms

  • The directional rule (composite stacks > parents) held but the magnitude (+0.10 → 0.00) didn’t survive. The qualitative direction of the pairwise-rule prediction remains correct; the quantitative magnitude was N=10 noise.

  • The variance discipline from PR #656 (vol-regime-filter-mean-vs-variance) was right: ranking by mean alone is misleading when all arms cluster within stdev. The 50-seed run confirms baseline and three_clock_vol_weighted are statistically equivalent.

The refined discipline rule

PR #710 / #742 / #748’s pairwise rule was about which composites will stack vs interfere. The N=50 result adds:

A composite that DIRECTIONALLY stacks (mean > both parents) may not statistically stack — the directional rule has sample-size cost. At N ≤ 20, +0.1 Sharpe differences are within noise; need N ≥ 50 to start distinguishing.

A corollary: before claiming a composite “works,” re-run at the highest seed count your time budget allows. The 1-second-per-seed harness can run N=100 in 100 seconds; the extra precision is worth the wait.

What this doesn’t show

  • Not that the composite is bad. Tying baseline at N=50 is a real result. It means the composite catches roughly the same total signal as baseline — same Sharpe at the same gross leverage. The operator’s choice between the two is about the variance shape (composite stdev 1.149 vs baseline 1.065 — slightly wider) or the worst-case (composite min −2.337 vs baseline −1.091 — composite is worse).

  • Not that the pairwise rule failed. The rule predicted directional stacking. At N=10 the data was consistent with the prediction; at N=50 it’s noisier. The rule’s cheap-pre-test property still holds — operators just need to set the bar at “directionally stacks” not “statistically stacks.”

  • Not that synthetic ≈ real. The real-data run is still the next required step. The N=50 synthetic result says “the composite ties baseline on synthetic”; real markets might produce a different rank order.

The next experiments

The session-summary v2’s three follow-ups remain:

  1. Higher seed count for three_clock_vol_weighted ← DONE here. Ties baseline at N=50.

  2. Lowest-correlation composite (three_clock_vol_regime × ts_momentum at +0.27). Whether stacking continues into the under-0.3 band is unknown. This is now the highest-priority follow-up.

  3. Real-data run. Still required.

deskgdfindgfstrategiesgsfeaturesgepromotionsgpapigabotgbwritinggw

Shortcuts are no-ops while typing in an input or textarea.