diff options
| author | Yuren Hao <yurenh2@illinois.edu> | 2026-07-10 13:11:32 -0500 |
|---|---|---|
| committer | Yuren Hao <yurenh2@illinois.edu> | 2026-07-10 13:11:32 -0500 |
| commit | dca5efea2f9f242cec4273a9e566e2694746d87c (patch) | |
| tree | 3e09b254ecea7390b216bbaf56679c87545f7f46 | |
| parent | 5d092bc602a97402cabb5580b4f1ca29d0c4bca9 (diff) | |
RESULT 8 prelim: Muon+beta_floor on EP = 1.7316 (-0.098 vs AdamW, n=1, zero skips); retraction closed; BP+Muon control + s2 launched
| -rw-r--r-- | docs/campaign/CASCADE_ABLATION_PLAN.md | 11 |
1 files changed, 11 insertions, 0 deletions
diff --git a/docs/campaign/CASCADE_ABLATION_PLAN.md b/docs/campaign/CASCADE_ABLATION_PLAN.md index fd93098..82e214d 100644 --- a/docs/campaign/CASCADE_ABLATION_PLAN.md +++ b/docs/campaign/CASCADE_ABLATION_PLAN.md @@ -397,3 +397,14 @@ skips=(d/g/r) through the 10-14k window. running on timan107 GPUs 6/2/7. NOTE farm-vs-local init differs (torch 2.3.1 vs 2.10 CUDA RNG) — config-matched anchors, not init-twins. - Wall watch armed on the epoch: report at step 14000 (past the old 11.4k wall) with skips=(d/g/r). + +### RESULT 8-PRELIM (2026-07-10 12:4x): Muon retraction CLOSED — with beta-floor, EP+Muon WINS big (n=1). +ol_ep_muon_s1 (OLMo2 + beta_floor + kretry + Muon, Pascal GPU7): **1.7316**, cos 0.9992, ZERO skips. +vs same-config-seed AdamW columns: BP 1.8294 / EP 1.8294 -> **-0.098 CE** (3-5x the metric noise band). +The original "naive Muon-on-EP fails (2.7515)" was ENTIRELY the missing beta-floor (audit correction +vindicated). Muon's known small/mid-scale advantage over AdamW TRANSFERS to EP gradients. +Controls launched: ol_bp_muon_s1 (the fair Muon-column comparison) + ol_ep_muon_s2 (seed robustness). +If BP+Muon lands ~1.73 too -> Muon helps both equally (parity preserved, recipe upgraded for BOTH +columns). If BP+Muon ~1.83 -> EP-specific synergy (bigger story, needs replication before claiming). +Interim: BP-twin epochs healthy at ~13.7k (best ~1.60 — already past the old-arch EP wall step); +local EP epoch at 4.8k, best 1.8148, zero skips, 1.97 it/s. |