diff options
Diffstat (limited to 'docs/campaign')
| -rw-r--r-- | docs/campaign/CASCADE_ABLATION_PLAN.md | 16 |
1 files changed, 16 insertions, 0 deletions
diff --git a/docs/campaign/CASCADE_ABLATION_PLAN.md b/docs/campaign/CASCADE_ABLATION_PLAN.md index ea3b40f..fd93098 100644 --- a/docs/campaign/CASCADE_ABLATION_PLAN.md +++ b/docs/campaign/CASCADE_ABLATION_PLAN.md @@ -381,3 +381,19 @@ diagnostics were all on the OLD arch) -> (2) lr channel (lower peak / faster dec mid-training danger window) -> (3) true contractivity control (damped-fb gamma<1 / cascade-jacreg) if OLMo2 still hits the wall.** Stage-1 OLMo2 epoch (parity-gated autolaunch) is the live test; watch skips=(d/g/r) through the 10-14k window. + +### RESULT 7 (2026-07-10 12:0x): OLMo2 4k PARITY — gate PASSED; arch worth ~0.07-0.10 CE to BOTH; epoch auto-launched. +- **BP+OLMo2: 1.8294/1.8333/1.8378 = 1.8335 (±0.004)** | **EP+OLMo2: 1.8294/1.8818/1.8817 = 1.8643** | + gap +0.031 -> PASS (<=0.05) -> stage1_ol_ep AUTO-LAUNCHED (58.8k steps, qk+floor+kretry+cosine+wd). +- OLMo2 improved BOTH columns vs old arch at 4k (BP 1.9066->1.8335; EP 1.8868->1.8643) — the arch + upgrade pays for itself immediately. +- HONEST READ: EP s1 == BP s1 to 4 decimals (1.8294, twin init); but EP s2/s3 trail their BP twins by + ~0.045. Mean gap +0.031 is WITHIN the best-of-noisy-val metric band (~0.02-0.03, per audit), so: + parity within noise, point estimate now slightly BP-leaning (was EP-leaning on old arch). Watch, not + act: candidate causes = beta_t schedule now keyed to untied W_out sigma; norm-after changing fb + conditioning (canary cos 0.9991 vs 1.0000). If the epoch shows a real gap, revisit. +- Pascal canaries GREEN (EP cos 0.9991 flat, 0 skips, 0.64 it/s; BP 2.0 it/s) -> farm UNBANNED for + cascade: 2x BP-twin epochs (stage1_ol_bp_s1/s2, ~8h) + Muon-with-floor retest (ol_ep_muon_s1) now + running on timan107 GPUs 6/2/7. NOTE farm-vs-local init differs (torch 2.3.1 vs 2.10 CUDA RNG) — + config-matched anchors, not init-twins. +- Wall watch armed on the epoch: report at step 14000 (past the old 11.4k wall) with skips=(d/g/r). |