diff options
| author | Yuren Hao <yurenh2@illinois.edu> | 2026-07-10 17:05:44 -0500 |
|---|---|---|
| committer | Yuren Hao <yurenh2@illinois.edu> | 2026-07-10 17:05:44 -0500 |
| commit | b3d05591ccc9fdc5252a1cf8f7ecff23bd4402fd (patch) | |
| tree | 37e04b5806d67a1b6c403a9ffa6d0efc3829ca55 /docs/campaign | |
| parent | 0520c4c708bcd2fcae0d73678cec6abfa7bedfcf (diff) | |
RESULT 8 sealed (Muon 3v3: BP 1.7098 / EP 1.7314); beta-floor dose-response @25k: 3e-4/5e-4/1e-3 -> cos 0.984/0.989/0.994, stable at 1e-3; Stage-2 recipe = late floor ramp
Diffstat (limited to 'docs/campaign')
| -rw-r--r-- | docs/campaign/CASCADE_ABLATION_PLAN.md | 12 |
1 files changed, 12 insertions, 0 deletions
diff --git a/docs/campaign/CASCADE_ABLATION_PLAN.md b/docs/campaign/CASCADE_ABLATION_PLAN.md index f992e68..2314e8e 100644 --- a/docs/campaign/CASCADE_ABLATION_PLAN.md +++ b/docs/campaign/CASCADE_ABLATION_PLAN.md @@ -440,3 +440,15 @@ reintroduces Adam/data confounds; cosine-LR shrinks late steps anyway). The BP e measurement: EP final within ~0.03 of BP -> erosion harmless; 0.1 behind -> quantified problem with a ready dial (late beta_floor schedule, e.g. 5e-4 past 20k). Pre-validation probe queued: when the farm frees, run ckpt-25000 + floor 5e-4 x 2k steps. Watcher re-armed at cos<0.96. + +### RESULT 8-SEALED + beta-floor dose-response (2026-07-10 16:1x). +**Muon 3v3 SEALED: BP+Muon 1.7020/1.7137/1.7136 = 1.7098 | EP+Muon 1.7316/1.7191/1.7435 = 1.7314.** +Muon default for both columns from Stage-2. The +0.02-0.03 BP-lean now CONSISTENT across two optimizer +columns (6v6) -> upgraded from noise to "probably real small effect"; primary suspect = late-training +SNR (see below), because it is beta-liftable: +**beta_floor dose-response @ckpt-25000 (same weights/batch): 3e-4 -> cos 0.984 | 5e-4 -> 0.9889 | +1e-3 -> 0.9940.** Raising the floor lifts cos exactly as the SNR mechanism predicts, with drift=0.000 +and zero skips at 1e-3 (larger nudge does NOT destabilize the OLMo2 relaxation). 2k-step traces +harvesting (auto-kill at 27k). RECIPE UPDATE for Stage-2 (and the next epoch): late beta_floor +schedule — floor 3e-4 early, ramp to ~1e-3 in the back half (or floor ∝ 1/grad-norm). This likely +also closes the +0.02-0.03 gap. |