From 75ff326dcb40cd960abd56e9c9c18a45d9e5e2c2 Mon Sep 17 00:00:00 2001 From: Yuren Hao Date: Sun, 5 Jul 2026 05:09:36 -0500 Subject: =?UTF-8?q?--sdpa:=20fused=20flash=20attention=20in=20the=20no=5Fg?= =?UTF-8?q?rad=20relax=20loop=20=E2=80=94=201.45x=20free=20phase,=20z*=20p?= =?UTF-8?q?arity=204e-7?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Scoped via blk._sdpa set only inside relax()'s loop (grad paths jvp/vjp/resreg keep the manual attention: no forward-mode-through-flash risk). Combined with --holofast: ~1.51x full-step exact-math tier. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn --- ep_run/sdpa_gate.log | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 ep_run/sdpa_gate.log (limited to 'ep_run/sdpa_gate.log') diff --git a/ep_run/sdpa_gate.log b/ep_run/sdpa_gate.log new file mode 100644 index 0000000..88bb685 --- /dev/null +++ b/ep_run/sdpa_gate.log @@ -0,0 +1,4 @@ +manual: res=9.981e+00 val=3.1589 relax150=1.650s + sdpa: res=9.981e+00 val=3.1056 relax150=1.140s +z* rel-diff=3.99e-07 speed=1.45x +SDPA_GATE_DONE -- cgit v1.2.3