diff options
Diffstat (limited to 'docs/campaign')
| -rw-r--r-- | docs/campaign/CASCADE_ABLATION_PLAN.md | 24 |
1 files changed, 24 insertions, 0 deletions
diff --git a/docs/campaign/CASCADE_ABLATION_PLAN.md b/docs/campaign/CASCADE_ABLATION_PLAN.md index 55df056..2cc9f05 100644 --- a/docs/campaign/CASCADE_ABLATION_PLAN.md +++ b/docs/campaign/CASCADE_ABLATION_PLAN.md @@ -342,3 +342,27 @@ val); the D1a "no-death" correction; Delta cancellation scope. (lr, sharpening-rate driver) and D (BP, EP-specificity). 8. Process fixes: watcher was not harness-tracked (user caught it — now all watchers via tracked bg tasks); zsh $VAR word-splitting cost two launch retries (all launches now via bash scripts). + +### RESULT 6 (2026-07-10 09:35): WALL-2 DIAGNOSED — marginal under-convergence, EP-specific; kretry fix shipped; OLMo2 matrix launched. +A/B/C/D verdict (resume from pre-bifurcation ckpt-10000, beta floored): +| arm | skips @ window | note | +|---|---|---| +| A ctl (K3, lr1e-3) | **16, accelerating** (val wobble 2.00@12400) | leading indicator REPRODUCES | +| B K8 | **2** | rejections nearly eliminated | +| C lr3e-4 | **1**, best 1.5163 (best of all) | never touches the edge | +| D BP (same ckpt/config/lr) | clean through 12750 | **EP-specific confirmed** | +**Mechanism (two walls, two levers — revises "K refuted"):** +- Wall-1 (~2-4k): cos erosion = finite-beta SNR -> beta-floor (K genuinely irrelevant there). +- Wall-2 (~11k+): operator sharpens -> a growing fraction of batches sit at the CONTRACTIVITY EDGE of + the nudged fb relaxation and under-converge at K3 -> drift-guard rejections climb -> gradient bias + + occasional marginal escapes -> blowup. K8 CONVERGES those batches (16 -> 2 rejections) => marginal + under-convergence, NOT hard divergence. lr modulates when the edge arrives (C: skips~1 and better CE). + BP has no relaxation -> no wall-2 (D clean). Original 12100 didn't literally replay in A (fresh Adam + + different data order — the recorded confounds) but the leading indicator did. +**FIX SHIPPED: `--kretry N`** — on drift-reject, RETRY the batch once with N fb rounds (B proved K8 +converges them) instead of dropping it. Converts biased skips into converged gradients; costs extra +rounds ONLY on marginal batches (~0.1-1% of steps). Telemetry: skips=(d/g/r). +**OLMo2 4k matrix LAUNCHED** (ol_bp_s1-3 + ol_ep_s1-3, wd 0.1, EP: beta_floor 3e-4 + kretry 8; twin +step-0 losses bitwise-identical per seed). Watcher auto-computes parity and — if EP mean within 0.05 +of BP — AUTO-LAUNCHES the Stage-1 OLMo2 TinyStories epoch (stage1_ol_ep, 58.8k steps, kretry armed). +OLMo2's bounded-per-branch signals may also shift wall-2 later; kretry is the belt-and-suspenders. |