diff options
| author | Yuren Hao <yurenh2@illinois.edu> | 2026-07-24 18:52:12 -0500 |
|---|---|---|
| committer | Yuren Hao <yurenh2@illinois.edu> | 2026-07-24 18:52:12 -0500 |
| commit | 27560970e2d967e62d22b8693811d7301fbc290d (patch) | |
| tree | eefd61a1eedfcc426f0611608e3ddedc5db5094c | |
| parent | c47abf122a2074250ff565b41f3cb3d47bf9b512 (diff) | |
RESULT 60: s345000走廊空(end075更糟3.4805,LR嫌疑排除)→闭合点275-300k,s250000=最后健康点; --res_gate宽度可配+bsign×beta_simple符号抹除bug修复(COMBO_CERT_PASS); fw135m_ratchet从s250000发射(bsign+棘轮+res_gate 1e-3,27h)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
| -rw-r--r-- | docs/campaign/CASCADE_ABLATION_PLAN.md | 22 | ||||
| -rw-r--r-- | ep_run/casc_eq_train.py | 7 |
2 files changed, 28 insertions, 1 deletions
diff --git a/docs/campaign/CASCADE_ABLATION_PLAN.md b/docs/campaign/CASCADE_ABLATION_PLAN.md index a0c8efb..5bb1d73 100644 --- a/docs/campaign/CASCADE_ABLATION_PLAN.md +++ b/docs/campaign/CASCADE_ABLATION_PLAN.md @@ -568,6 +568,28 @@ direction), not training-under-fault; wave-2 = co-training with faults injected +### RESULT 60 (2026-07-24): s345000 CORRIDOR IS EMPTY (end075 3.4805 < original, worse); +closure located at ~275-300k; CONTINUATION = bsign+ratchet+res_gate from s250000. Two code +fixes en route (sign-wipe combo bug + configurable gate), both fire-certified. +end075 (beta 7.5e-4 from s345000): best 3.4805, val climbing to 3.9, gn doubled — LOWER beta +did WORSE than the original 3e-3 over the same segment. LR-scheduler suspect CLEARED (resume +fast-forward exists, line 813). Diagnosis: at s345000 the corridor is EMPTY — ceiling ~1e-3 +(contraction) while the state's elevated relax-noise floor (resK stuck 2.6e-4 at ALL beta, +100x above healthy) pushes the effective lower bound UP past it. Over-ceiling operation bakes +in state damage (the 72M simple-lineage lesson, width edition). +CLOSURE PROBE (resK-to-floor reading, rho-noise verdicts discounted): s250000 CLEAN at 3e-3 +(resK 1.3e-6, ceiling ~5e-3) | s300000 boundary (3e-3 resK 2.2e-5) | s345000 dead. Crossing +~275-300k. Last healthy resume point = s250000. +FIXES: (1) --res_gate flag (0.02 was a C512 constant; C768 ran 100k steps semi-converged under +it; per-width rule ~100x healthy K=3 floor -> 1e-3 for C768); (2) bsign+beta_simple SIGN-WIPE +bug — the beta_simple override sat after the sign flip and silently disabled bsign when both +were on (never manifested: 72M ratchet ran plain). Flip moved after the override. +COMBO_CERT_PASS: forced-knock smoke shows sign flips + knocks + persistent halving together. +IN FLIGHT: fw135m_ratchet — resume s250000, bsign + halve-only ratchet + res_gate 1e-3, +beta0 3e-3, 3-GPU, 190k-step replay (~27h). Expected: knocks track the sinking ceiling +(~290k, ~1.5-2e-3, endgame ~1e-3), anneal harvested inside the corridor. Verdict = best vs +3.3839 (bsign fixed-3e-3) and vs 72M band 3.364. + ### RESULT 59b (2026-07-24): C768 CEILING MEASURED — the run spent its second half 2-3x ABOVE the ceiling, and the ceiling sinks UNDER BSIGN at this width. R59's "graze" and "stability transfers" both CORRECTED. diff --git a/ep_run/casc_eq_train.py b/ep_run/casc_eq_train.py index eb64082..60b6113 100644 --- a/ep_run/casc_eq_train.py +++ b/ep_run/casc_eq_train.py @@ -49,6 +49,10 @@ ap.add_argument('--bsign_rand', action='store_true') # random-sign beta per ste ap.add_argument('--bf16', action='store_true') # cast model to bf16 (E-accumulation + tok_sigma stay fp32) — the x0.5 cost lever, GATE before production ap.add_argument('--amp', action='store_true') # PROPER mixed precision: autocast(bf16) matmuls, fp32 params/states/d/E — amp_gate.py PASSED 2026-07-12 (cos 0.9682 vs fp32 0.9687); --bf16 naive-cast stays DEAD (state quantization, RESULT 11) ap.add_argument('--dtop_every', type=int, default=1) # 1 = exact (DEFAULT, BP-parity); 2 = fast mode (~20% cheaper, ~4% CE tax at high lr) +ap.add_argument('--res_gate', type=float, default=0.02) # legality residual threshold; 0.02 was calibrated + # at C512 — C768 ran half a schedule semi-converged + # UNDER it (R59b). Per-width rule: ~100x the healthy + # K=3 residual floor measured by the rho probe. ap.add_argument('--watch_every', type=int, default=2000) # wandb-only telemetry cadence: act/weight RMS ap.add_argument('--gate_every', type=int, default=200) # in-training cos(EP,BP) telemetry; <=0 = fully BP-free (no bp_gate at all) ap.add_argument('--gate_govern', action='store_true') # let gate cos adjust K/bscale (default: observe-only => training control is BP-free) @@ -503,6 +507,7 @@ def ep_step(x, y): if args.beta_simple >= 1.0: # FULL ownership: no sigma-scaling, no floor/bf_late, no cap — beta = start * measured multiplier beta_t = args.beta * GOV.get('bsimp', 1.0) + # sign flip LAST — the beta_simple override must not wipe it (bsign+ratchet combo fix) if args.bsign_rand and torch.rand((), generator=BGEN).item() < 0.5: beta_t = -beta_t EST = args.est if args.est_late and GOV['step'] >= args.est_late_at: EST = args.est_late @@ -552,7 +557,7 @@ def ep_step(x, y): # ~1 (the documented v1-cap starvation bug; Codex's OR resurrected it, the SV2 # arm caught it: 6-halving storm every step, beta pinned at 4.7e-5). The 35k # runaway had res >> 0.02 with rho < 0.9 — res-alone provably plugs that hole. - if (not math.isfinite(res_s)) or res_s > 0.02: + if (not math.isfinite(res_s)) or res_s > args.res_gate: return False return True _ok0 = _legal(gdrift) |
