diff options
| author | Yuren Hao <yurenh2@illinois.edu> | 2026-07-10 15:38:46 -0500 |
|---|---|---|
| committer | Yuren Hao <yurenh2@illinois.edu> | 2026-07-10 15:38:46 -0500 |
| commit | 0520c4c708bcd2fcae0d73678cec6abfa7bedfcf (patch) | |
| tree | bdc1c66e4e32a8a2c1b965cbbce3963cebf0070e /docs/campaign/CASCADE_ABLATION_PLAN.md | |
| parent | 35f9d6e8fea9cfbd4fc235ec00135564f93b7465 (diff) | |
Watch: late-epoch cos erosion = intrinsic late-training SNR decline; let it run, BP twins are the measurement; floor-probe queued
Diffstat (limited to 'docs/campaign/CASCADE_ABLATION_PLAN.md')
| -rw-r--r-- | docs/campaign/CASCADE_ABLATION_PLAN.md | 10 |
1 files changed, 10 insertions, 0 deletions
diff --git a/docs/campaign/CASCADE_ABLATION_PLAN.md b/docs/campaign/CASCADE_ABLATION_PLAN.md index 9cbefda..f992e68 100644 --- a/docs/campaign/CASCADE_ABLATION_PLAN.md +++ b/docs/campaign/CASCADE_ABLATION_PLAN.md @@ -430,3 +430,13 @@ column's +0.031 (consistent slight BP-lean on OLMo2, noise-band edge, on the wat HW-narrative guard: Muon's Newton-Schulz is matrix-matrix (analog-dead) but the optimizer lives DIGITAL-side per standing doctrine — GPU-pretraining Muon does NOT conflict with the factored-Adam analog training story. Filling to 3v3 (BP+Muon s2/s3, EP+Muon s3) for the seal. + +### WATCH (2026-07-10 14:0x): late-epoch cos erosion = intrinsic late-training SNR decline. Decision: let it run. +cos 0.9894@17k -> 0.9840@22k (-0.0011/1k), ZERO skips, gn calm, val still improving (1.6353). sigma +plateaued (~32) and beta at floor => ratio stable => NOT the sigma-growth wall-1. Mechanism: true +gradient magnitude shrinks as CE approaches optimum while the estimator noise floor stays constant -> +SNR falls with the signal. Extrapolates to cos~0.94 by 58.8k. DECISION: no mid-flight surgery (resume +reintroduces Adam/data confounds; cosine-LR shrinks late steps anyway). The BP epoch twins ARE the +measurement: EP final within ~0.03 of BP -> erosion harmless; 0.1 behind -> quantified problem with a +ready dial (late beta_floor schedule, e.g. 5e-4 past 20k). Pre-validation probe queued: when the farm +frees, run ckpt-25000 + floor 5e-4 x 2k steps. Watcher re-armed at cos<0.96. |