diff options
| author | Yuren Hao <yurenh2@illinois.edu> | 2026-07-06 09:42:52 -0500 |
|---|---|---|
| committer | Yuren Hao <yurenh2@illinois.edu> | 2026-07-06 09:42:52 -0500 |
| commit | 40be67d4f5b5a6b46c662c70b759e585e136d70e (patch) | |
| tree | bbe02d5164c9b86a0af6a3634fb7556f065c8672 /ep_run/fastfp_gate.py | |
| parent | 9a8b2796ca12e4d4c24717485a635a301aa6d07f (diff) | |
Tier-3 gates: Anderson math-yes/impl-no (parked for v2); bf16polish UNSAFE near-edge (eval-only); dp_ep.py ready
Anderson: res 25-35x deeper per budget but 5.8x slower (naive history stacks
+ per-iter safeguard eval) — v2 = ring buffers + periodic safeguard, est +1.3x
on the speed tier. bf16+20polish: 1.41x free phase, res parity, BUT z-diff
1.2e-3 — near-marginal operators contract too slowly for a 20-step polish
(0.998^20≈0.96), same magnitude as the TF32 kill verdict and the estimator's
50%-sensitivity input. Predicted by our own depth/noise theory. Flags kept
with warnings; neither ships for training. dp_ep.py: manual-allreduce EP DP
(controller in lockstep, aligned collectives), smoke pending freed 1080s.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
Diffstat (limited to 'ep_run/fastfp_gate.py')
| -rw-r--r-- | ep_run/fastfp_gate.py | 40 |
1 files changed, 40 insertions, 0 deletions
diff --git a/ep_run/fastfp_gate.py b/ep_run/fastfp_gate.py new file mode 100644 index 0000000..f65f71f --- /dev/null +++ b/ep_run/fastfp_gate.py @@ -0,0 +1,40 @@ +"""Gate for --fastfp (Anderson z*) and --bf16polish: endpoint parity + residual + eval-count/time vs the +150-step Euler reference, on two operators (deeply-trained fast-adaptive + near-edge s2000 — the hard +case: Anderson historically fails on cycling ops, s2000's marginal band is the stress test). +GPU shared with abl_pair — timings are relative, parity/evals exact.""" +import time, torch +import lt_ep_train as L + +for name, path in (('s2000', 'runs/redx_traj/s2000.pt'), ('fast-adaptive', 'runs/ep_fast_adaptive.pt')): + torch.manual_seed(0) + blk = L.EQBlock(512, 16, 256, 256, c=1.0, attn_mode='thick'); blk.qknorm = True + ck = torch.load(path, map_location=L.dev) + with torch.no_grad(): + for p, w in zip(blk.allp, ck['allp']): + p.copy_(w.to(L.dev)) + torch.manual_seed(42) + idx, _ = L.get_batch('train', 24, 256) + xin = blk.embed(idx).detach() + + torch.cuda.synchronize(); t = time.time() + z_ref = L.relax(blk, xin.clone(), xin, 150, 0.1) + torch.cuda.synchronize(); t_ref = time.time() - t + r_ref = (L.relax(blk, z_ref, xin, 1, 0.1) - z_ref).norm().item() + print(f"[{name}] euler150 : {t_ref:5.2f}s res={r_ref:.2e}", flush=True) + + torch.cuda.synchronize(); t = time.time() + z_aa, evals = L.anderson_relax(blk, xin.clone(), xin, 150, 0.1) + torch.cuda.synchronize(); t_aa = time.time() - t + r_aa = (L.relax(blk, z_aa, xin, 1, 0.1) - z_aa).norm().item() + zd = ((z_aa - z_ref).norm() / (z_ref.norm() + 1e-12)).item() + print(f"[{name}] anderson : {t_aa:5.2f}s res={r_aa:.2e} evals={evals} z-diff={zd:.2e}", flush=True) + + blk.bf16polish = 20 + torch.cuda.synchronize(); t = time.time() + z_bf = L.relax(blk, xin.clone(), xin, 150, 0.1) + torch.cuda.synchronize(); t_bf = time.time() - t + blk.bf16polish = 0 + r_bf = (L.relax(blk, z_bf, xin, 1, 0.1) - z_bf).norm().item() + zdb = ((z_bf - z_ref).norm() / (z_ref.norm() + 1e-12)).item() + print(f"[{name}] bf16+20 : {t_bf:5.2f}s res={r_bf:.2e} z-diff={zdb:.2e}", flush=True) +print("FASTFP_GATE_DONE", flush=True) |
