diff options
Diffstat (limited to 'docs/campaign')
| -rw-r--r-- | docs/campaign/CASCADE_ABLATION_PLAN.md | 10 |
1 files changed, 10 insertions, 0 deletions
diff --git a/docs/campaign/CASCADE_ABLATION_PLAN.md b/docs/campaign/CASCADE_ABLATION_PLAN.md index f3b9f87..61860f6 100644 --- a/docs/campaign/CASCADE_ABLATION_PLAN.md +++ b/docs/campaign/CASCADE_ABLATION_PLAN.md @@ -528,3 +528,13 @@ stage1b_f3e3cont (floor 3e-3 from s45000): **1.2883** vs stage1b 1.2808 (floor 1 competitive at full ImageNet by Kerjan-Hoier-Scellier) then centered (2x nudge) if needed; (b) fb K=3 finite-relaxation bias; (c) Muon x gradient-noise interaction; (d) ~0.02-0.03 of the 0.05 is metric-noise band. Recipe note: random-sign is a one-line trainer change (sign of beta_t per step). + +### RESULT 13 (2026-07-12): bsign (random-sign beta) NEUTRAL at 4k — 42M gap-chasing has hit the noise floor. THREAD CLOSED. +bsign 3-seed: 1.7031/1.7562/1.7552 (mean 1.7382) vs single-sided 1.7314 vs BP+Muon 1.7098. The bias +reduction is cancelled by injected update-direction variance at this horizon (seed spread now dominates: +s1 alone beat the BP mean). Ledger of the residual-0.05 epoch gap after three probes: NOT late-beta-SNR +(f3e3cont), NOT K (K5 probe), NOT first-order sign bias at short horizon (bsign). Remaining mass: +~0.02-0.03 metric-noise band + small unattributed accumulation. **Decision: stop polishing 42M.** +Carry `--bsign_rand` and a future centered mode as Stage-2 A/B flags; the gap question re-opens at +300M/real-corpus where it means something. Effort pivots to: (1) Stage-2 data pipeline (FineWeb-Edu + +32k tokenizer), (2) E-tier tolerance suite on the idle farm (hardware track / UIUC outreach feed). |