From 9c2d7f74d30c637d55b0b34fb5fe830318c0b47c Mon Sep 17 00:00:00 2001 From: Yuren Hao Date: Fri, 10 Jul 2026 11:13:40 -0500 Subject: RESULT 6 addendum: K8 also blew at 12600 -> wall-2 is true contractivity crossing (K delays only); kretry demoted to mitigation; defense = OLMo2 arch > lr channel > damped-fb/jacreg --- docs/campaign/CASCADE_ABLATION_PLAN.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) (limited to 'docs/campaign/CASCADE_ABLATION_PLAN.md') diff --git a/docs/campaign/CASCADE_ABLATION_PLAN.md b/docs/campaign/CASCADE_ABLATION_PLAN.md index 2cc9f05..ea3b40f 100644 --- a/docs/campaign/CASCADE_ABLATION_PLAN.md +++ b/docs/campaign/CASCADE_ABLATION_PLAN.md @@ -366,3 +366,18 @@ rounds ONLY on marginal batches (~0.1-1% of steps). Telemetry: skips=(d/g/r). step-0 losses bitwise-identical per seed). Watcher auto-computes parity and — if EP mean within 0.05 of BP — AUTO-LAUNCHES the Stage-1 OLMo2 TinyStories epoch (stage1_ol_ep, 58.8k steps, kretry armed). OLMo2's bounded-per-branch signals may also shift wall-2 later; kretry is the belt-and-suspenders. + +### RESULT 6-ADDENDUM (2026-07-10 10:5x): B(K8) ALSO BLEW at matched step — K delays, does NOT prevent. +diag_B_k8 @12600: train 3.37 / val 3.63 (best 1.6755 pre-blowup), gn 18.6, skips 2->14. So wall-2 is +NOT merely marginal under-convergence: the nudged fb iteration becomes GENUINELY DIVERGENT for a +growing batch fraction as the operator sharpens (true contractivity crossing — the cascade Hopf wall). +More rounds converge the marginal shell only; once past the edge no K helps. **kretry DEMOTED from fix +to mitigation** (still right for sporadic healthy-regime rejections). Note also: B blew with only 14 +total rejections => most bad gradients passed UNDER the drift-0.5 threshold (loose guard + gn-EMA +poisoning during degradation). +Surviving facts: C (lr 3e-4) clean at 12600 (skips=1) -> sharpening RATE is the driver; D (BP) clean +-> EP-specific. **Defense ranking now: (1) OLMo2 arch (different operator: bounded branches + QK-norm; +diagnostics were all on the OLD arch) -> (2) lr channel (lower peak / faster decay through the +mid-training danger window) -> (3) true contractivity control (damped-fb gamma<1 / cascade-jacreg) if +OLMo2 still hits the wall.** Stage-1 OLMo2 epoch (parity-gated autolaunch) is the live test; watch +skips=(d/g/r) through the 10-14k window. -- cgit v1.2.3