diff options
Diffstat (limited to 'docs/campaign')
| -rw-r--r-- | docs/campaign/CASCADE_ABLATION_PLAN.md | 37 |
1 files changed, 37 insertions, 0 deletions
diff --git a/docs/campaign/CASCADE_ABLATION_PLAN.md b/docs/campaign/CASCADE_ABLATION_PLAN.md index 19f97bb..55df056 100644 --- a/docs/campaign/CASCADE_ABLATION_PLAN.md +++ b/docs/campaign/CASCADE_ABLATION_PLAN.md @@ -305,3 +305,40 @@ geta<1 damping), NOT final_ln. A=control (K3,lr1e-3) -> should reproduce skip-climb+blowup; B=K8 (does more fb rounds hold skips? = marginal-contractivity test); C=lr3e-4 (slower sharpening -> delayed edge? = driver test). Code added: --resume, --sig0, --final_ln, --qk_norm(CausalSelfAttn/SDPA). Watcher diag_watch.sh armed. + +## AUDIT (2026-07-10, model switch): re-review of the day's conclusions. Corrections + added controls. +**What SURVIVES audit:** RESULT 1 (K-invariance data is solid; K plumbed, paid wall-clock, identical +cos/CE); RESULT 2 (beta-floor effect is decisive and mechanistic: floored arms pin cos, unfloored +collapses); the 4k-horizon numbers themselves; the blowup telemetry read (skips lead gn lead cos lead +val); the D1a "no-death" correction; Delta cancellation scope. +**CORRECTIONS from audit:** +1. **Muon verdict RETRACTED as confounded.** d1_ep_muon (2.7515, cos 0.82) ran in the ORIGINAL D1a + batch, i.e. WITHOUT beta_floor — its cos collapse mirrors the unfloored control (0.896). "Naive + Muon-on-EP fails" is NOT established; needs a re-run with beta_floor before any conclusion. +2. **Parity claims toned down.** n=3 with best-of-noisy-val (6-batch val, min over ~500 evals -> + selection bias ~0.02-0.03, applied to both arms) means "EP 1.8907 vs BP 1.9192" is PARITY with an + EP-leaning point estimate, not "EP beats BP". (RESULT 3's all-3-EP-below-all-3-BP is p~=0.05 rank + evidence — suggestive, not sealed.) Same for RESULT 4 (EP s2 1.9176 > BP best 1.8753). +3. **"Depth-tax FULLY removed" was premature** — true only at the 4k-step horizon; the epoch blowup at + ~11.4k shows a second, longer-horizon wall. Claim scoped accordingly. +4. **"skips = relaxation non-convergence" is UNVERIFIED.** The skips counter conflates the drift-guard + and the gn-EMA-guard; drift telemetry is stale-on-reject (GOV['drift'] not updated on drift-reject) + while gn telemetry does update on gn-reject. Guard-split counters (skd/skg) now added to the log + line for all future runs. The contractivity-bifurcation story remains the leading HYPOTHESIS, not + a finding. +5. **A/B/C lacked the decisive control: a BP arm.** If BP-from-the-same-ckpt ALSO blows up, the blowup + is a CONFIG instability (tied readout + NO final LayerNorm + sig~30 logits is genuinely nonstandard + — every real GPT has final-LN; final_ln then likely IS the fix, via bounded logits/curvature, even + though the sig->beta-SNR mechanism was refuted), and EP is exonerated. If BP sails through while A + blows, the bifurcation is EP-specific -> jacreg/damped-fb. **diag_D_bp launched** (BP + --resume + added to casc_bp_train, same ckpt-10000, qk_norm, lr 1e-3). +6. **Resume confounds now on record:** optimizer state is NOT in the ckpt (fresh Adam moments — sig + jumped 29.8->35.6 within 300 steps of resume, visibly faster drift than the original run) and the + data-order RNG restarts from the step-0 stream. So arm A can only reproduce the blowup + STATISTICALLY, not at step 12100; if ALL arms blow immediately after resume, suspect the + Adam-cold-start artifact rather than the original mechanism. +7. **Arm B (K8) is weakly informative by design:** for a genuinely divergent nudged iteration, MORE + rounds = MORE drift, so both "K8 helps" and "K8 hurts" fit the story. The causal weight is on C + (lr, sharpening-rate driver) and D (BP, EP-specificity). +8. Process fixes: watcher was not harness-tracked (user caught it — now all watchers via tracked bg + tasks); zsh $VAR word-splitting cost two launch retries (all launches now via bash scripts). |