diff options
| author | Yuren Hao <yurenh2@illinois.edu> | 2026-07-14 07:02:01 -0500 |
|---|---|---|
| committer | Yuren Hao <yurenh2@illinois.edu> | 2026-07-14 07:02:01 -0500 |
| commit | 5d2554c1e846a79cec1d76c7e8eaf3d0c2042227 (patch) | |
| tree | 05802755190d216673b6a98096acec53055fe4c4 | |
| parent | 643983aec870af4ba7e71eafc6c1ba37b2348730 (diff) | |
RESULT 18: wall-2 returns at 72M/FineWeb 195k (drift crossing, beta-stress confirmed by f3e3); runs killed, geta fix probes launched from s195000
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
| -rw-r--r-- | docs/campaign/CASCADE_ABLATION_PLAN.md | 20 |
1 files changed, 20 insertions, 0 deletions
diff --git a/docs/campaign/CASCADE_ABLATION_PLAN.md b/docs/campaign/CASCADE_ABLATION_PLAN.md index c044fb6..173cc77 100644 --- a/docs/campaign/CASCADE_ABLATION_PLAN.md +++ b/docs/campaign/CASCADE_ABLATION_PLAN.md @@ -564,6 +564,26 @@ direction), not training-under-fault; wave-2 = co-training with faults injected (per-row status updated) + UIUC outreach dossier. + +### RESULT 18 (2026-07-14): WALL-2 RETURNS AT SCALE — fw72m diverged at ~195k steps; f3e3 confirms beta-stress. +Timeline (fw72m, 72M/FineWeb/32k): healthy to 168k (drift 0.013-0.017, skips 0, best val 3.7117 +@~184k ~= 1.13B tokens); precursor drift SPIKE 0.044 @172k (transient); boundary contact 192-196k +(drift 0.055-0.069, first skips); MASS CROSSING @200k (drift pinned 0.46-0.475 vs guard 0.5, +skips +1000/4k = 25% reject rate, val 3.98 -> 5.5 divergence on the biased surviving subset). +**f3e3 branch (floor 3e-3 from 140k) crossed the SAME wall EARLIER and harder (drift rejects 14.7k, +train 27) — nudged displacement ~ beta => beta is the relaxation stress amplifier. The user's +"beta 大了不收敛" is this, live.** Both runs killed (ckpts intact to s215000; best-val assets +preserved). LESSONS: (1) the 42M/TinyStories architecture cure (norm placement) DELAYED the +crossing past 58.8k there but did NOT eliminate the mechanism — theta drift crossed at 195k in the +bigger/harder regime; "wall-2 ELIMINATED" is rescoped to "wall-2 delayed beyond horizon at 42M/TS". +(2) drift-creep + spike is the leading indicator (the aep-dynamics control-map discipline is now +LOAD-BEARING for the LLM line — first in-vivo transfer of the paper's machinery at scale). +(3) My blowup watchers were blind (awk field bug, $6='val' string) — fixed pattern: grep -oE. +FIX PROBES (running, from healthy s195000 through the full wall zone to 201k): w2_ctl (replicate), +w2_geta07 (damped mixing 0.7 = the principled contraction restorer), w2_geta05_k5, w2_blow +(floor back to 3e-4 = beta-stress direct test). Crown status: best ckpt 3.7117@1.13B tokens is a +trained artifact but the clean crown re-run waits for the fix verdict. + ### RESULT 17 (2026-07-13): fw72m LAUNCHED (the 62.7M-crown run) + size ladder complete + resume upgraded. - **fw72m**: 72.11M (L12 C512 @32k vocab) x 1.44B FineWeb-Edu tokens (Chinchilla), --amp, first NCCL 2-GPU DDP production run (GPU0+3, B12/rank = global B24 preserving recipe semantics). Startup: |