summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorYuren Hao <yurenh2@illinois.edu>2026-07-24 18:52:12 -0500
committerYuren Hao <yurenh2@illinois.edu>2026-07-24 18:52:12 -0500
commit27560970e2d967e62d22b8693811d7301fbc290d (patch)
treeeefd61a1eedfcc426f0611608e3ddedc5db5094c
parentc47abf122a2074250ff565b41f3cb3d47bf9b512 (diff)
RESULT 60: s345000走廊空(end075更糟3.4805,LR嫌疑排除)→闭合点275-300k,s250000=最后健康点; --res_gate宽度可配+bsign×beta_simple符号抹除bug修复(COMBO_CERT_PASS); fw135m_ratchet从s250000发射(bsign+棘轮+res_gate 1e-3,27h)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
-rw-r--r--docs/campaign/CASCADE_ABLATION_PLAN.md22
-rw-r--r--ep_run/casc_eq_train.py7
2 files changed, 28 insertions, 1 deletions
diff --git a/docs/campaign/CASCADE_ABLATION_PLAN.md b/docs/campaign/CASCADE_ABLATION_PLAN.md
index a0c8efb..5bb1d73 100644
--- a/docs/campaign/CASCADE_ABLATION_PLAN.md
+++ b/docs/campaign/CASCADE_ABLATION_PLAN.md
@@ -568,6 +568,28 @@ direction), not training-under-fault; wave-2 = co-training with faults injected
+### RESULT 60 (2026-07-24): s345000 CORRIDOR IS EMPTY (end075 3.4805 < original, worse);
+closure located at ~275-300k; CONTINUATION = bsign+ratchet+res_gate from s250000. Two code
+fixes en route (sign-wipe combo bug + configurable gate), both fire-certified.
+end075 (beta 7.5e-4 from s345000): best 3.4805, val climbing to 3.9, gn doubled — LOWER beta
+did WORSE than the original 3e-3 over the same segment. LR-scheduler suspect CLEARED (resume
+fast-forward exists, line 813). Diagnosis: at s345000 the corridor is EMPTY — ceiling ~1e-3
+(contraction) while the state's elevated relax-noise floor (resK stuck 2.6e-4 at ALL beta,
+100x above healthy) pushes the effective lower bound UP past it. Over-ceiling operation bakes
+in state damage (the 72M simple-lineage lesson, width edition).
+CLOSURE PROBE (resK-to-floor reading, rho-noise verdicts discounted): s250000 CLEAN at 3e-3
+(resK 1.3e-6, ceiling ~5e-3) | s300000 boundary (3e-3 resK 2.2e-5) | s345000 dead. Crossing
+~275-300k. Last healthy resume point = s250000.
+FIXES: (1) --res_gate flag (0.02 was a C512 constant; C768 ran 100k steps semi-converged under
+it; per-width rule ~100x healthy K=3 floor -> 1e-3 for C768); (2) bsign+beta_simple SIGN-WIPE
+bug — the beta_simple override sat after the sign flip and silently disabled bsign when both
+were on (never manifested: 72M ratchet ran plain). Flip moved after the override.
+COMBO_CERT_PASS: forced-knock smoke shows sign flips + knocks + persistent halving together.
+IN FLIGHT: fw135m_ratchet — resume s250000, bsign + halve-only ratchet + res_gate 1e-3,
+beta0 3e-3, 3-GPU, 190k-step replay (~27h). Expected: knocks track the sinking ceiling
+(~290k, ~1.5-2e-3, endgame ~1e-3), anneal harvested inside the corridor. Verdict = best vs
+3.3839 (bsign fixed-3e-3) and vs 72M band 3.364.
+
### RESULT 59b (2026-07-24): C768 CEILING MEASURED — the run spent its second half 2-3x
ABOVE the ceiling, and the ceiling sinks UNDER BSIGN at this width. R59's "graze" and "stability
transfers" both CORRECTED.
diff --git a/ep_run/casc_eq_train.py b/ep_run/casc_eq_train.py
index eb64082..60b6113 100644
--- a/ep_run/casc_eq_train.py
+++ b/ep_run/casc_eq_train.py
@@ -49,6 +49,10 @@ ap.add_argument('--bsign_rand', action='store_true') # random-sign beta per ste
ap.add_argument('--bf16', action='store_true') # cast model to bf16 (E-accumulation + tok_sigma stay fp32) — the x0.5 cost lever, GATE before production
ap.add_argument('--amp', action='store_true') # PROPER mixed precision: autocast(bf16) matmuls, fp32 params/states/d/E — amp_gate.py PASSED 2026-07-12 (cos 0.9682 vs fp32 0.9687); --bf16 naive-cast stays DEAD (state quantization, RESULT 11)
ap.add_argument('--dtop_every', type=int, default=1) # 1 = exact (DEFAULT, BP-parity); 2 = fast mode (~20% cheaper, ~4% CE tax at high lr)
+ap.add_argument('--res_gate', type=float, default=0.02) # legality residual threshold; 0.02 was calibrated
+ # at C512 — C768 ran half a schedule semi-converged
+ # UNDER it (R59b). Per-width rule: ~100x the healthy
+ # K=3 residual floor measured by the rho probe.
ap.add_argument('--watch_every', type=int, default=2000) # wandb-only telemetry cadence: act/weight RMS
ap.add_argument('--gate_every', type=int, default=200) # in-training cos(EP,BP) telemetry; <=0 = fully BP-free (no bp_gate at all)
ap.add_argument('--gate_govern', action='store_true') # let gate cos adjust K/bscale (default: observe-only => training control is BP-free)
@@ -503,6 +507,7 @@ def ep_step(x, y):
if args.beta_simple >= 1.0:
# FULL ownership: no sigma-scaling, no floor/bf_late, no cap — beta = start * measured multiplier
beta_t = args.beta * GOV.get('bsimp', 1.0)
+ # sign flip LAST — the beta_simple override must not wipe it (bsign+ratchet combo fix)
if args.bsign_rand and torch.rand((), generator=BGEN).item() < 0.5: beta_t = -beta_t
EST = args.est
if args.est_late and GOV['step'] >= args.est_late_at: EST = args.est_late
@@ -552,7 +557,7 @@ def ep_step(x, y):
# ~1 (the documented v1-cap starvation bug; Codex's OR resurrected it, the SV2
# arm caught it: 6-halving storm every step, beta pinned at 4.7e-5). The 35k
# runaway had res >> 0.02 with rho < 0.9 — res-alone provably plugs that hole.
- if (not math.isfinite(res_s)) or res_s > 0.02:
+ if (not math.isfinite(res_s)) or res_s > args.res_gate:
return False
return True
_ok0 = _legal(gdrift)