diff options
Diffstat (limited to 'docs/campaign/CASCADE_ABLATION_PLAN.md')
| -rw-r--r-- | docs/campaign/CASCADE_ABLATION_PLAN.md | 12 |
1 files changed, 12 insertions, 0 deletions
diff --git a/docs/campaign/CASCADE_ABLATION_PLAN.md b/docs/campaign/CASCADE_ABLATION_PLAN.md index f992e68..2314e8e 100644 --- a/docs/campaign/CASCADE_ABLATION_PLAN.md +++ b/docs/campaign/CASCADE_ABLATION_PLAN.md @@ -440,3 +440,15 @@ reintroduces Adam/data confounds; cosine-LR shrinks late steps anyway). The BP e measurement: EP final within ~0.03 of BP -> erosion harmless; 0.1 behind -> quantified problem with a ready dial (late beta_floor schedule, e.g. 5e-4 past 20k). Pre-validation probe queued: when the farm frees, run ckpt-25000 + floor 5e-4 x 2k steps. Watcher re-armed at cos<0.96. + +### RESULT 8-SEALED + beta-floor dose-response (2026-07-10 16:1x). +**Muon 3v3 SEALED: BP+Muon 1.7020/1.7137/1.7136 = 1.7098 | EP+Muon 1.7316/1.7191/1.7435 = 1.7314.** +Muon default for both columns from Stage-2. The +0.02-0.03 BP-lean now CONSISTENT across two optimizer +columns (6v6) -> upgraded from noise to "probably real small effect"; primary suspect = late-training +SNR (see below), because it is beta-liftable: +**beta_floor dose-response @ckpt-25000 (same weights/batch): 3e-4 -> cos 0.984 | 5e-4 -> 0.9889 | +1e-3 -> 0.9940.** Raising the floor lifts cos exactly as the SNR mechanism predicts, with drift=0.000 +and zero skips at 1e-3 (larger nudge does NOT destabilize the OLMo2 relaxation). 2k-step traces +harvesting (auto-kill at 27k). RECIPE UPDATE for Stage-2 (and the next epoch): late beta_floor +schedule — floor 3e-4 early, ramp to ~1e-3 in the back half (or floor ∝ 1/grad-norm). This likely +also closes the +0.02-0.03 gap. |