summaryrefslogtreecommitdiff
path: root/ep_run/casc_eq_train.py
AgeCommit message (Collapse)Author
2026-07-09A0.4 precision gate (TF32 harmless cos 0.9946==fp32; pure-bf16 cos 0.9427) + ↵Yuren Hao
Muon hybrid optimizer (--opt muon) wired into both trainers; D1a flagship matrix launched (L12xC512, 3xBP + 3xEP + Muon arms) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09casc_eq_train: exact mode (dtop_every=1) now DEFAULT — bisection showed ↵Yuren Hao
the v7 dedup cost ~4% CE at tuned lr; fast mode kept as opt-in. K3 verdict: exact equilibrium-EP 2.0481 vs BP 2.053+-0.004 = matched-tuning PARITY Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09casc_eq_train v7: two algorithmic dedups — d_top refreshed every other ↵Yuren Hao
round (O(beta^2) error), last-rebuild graphs reused by the theta-readout (kills one full graphed chain per step); fp32 untouched for fair BP twinning Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09casc_eq_train v6: graph-reuse fb (rebuild graphs feed next round's vjps), ↵Yuren Hao
--compile flag, tok-sigma amortized every 25 steps — 2.23 -> 2.68 it/s on 1080 (3.28x BP) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09cascade root cause found: tied readout + default N(0,1) embedding init = ↵Yuren Hao
pathological top-CE stiffness once predictions sharpen (sigma_tok~76 vs GPT-standard 0.02 giving ~1.6); add --untie and --tok_init; diag arms A/B/C Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09casc_eq_train v4: quality-governed estimator — periodic measured ↵Yuren Hao
cos(EP,BP) drives K/beta (spend when quality drops, relax when abundant); guards reduced to sanity-only Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09casc_eq_train v3: adaptive beta (tok-sigma stiffness compensation), ↵Yuren Hao
adaptive-K fb with contraction test, non-contraction bail, in-training cos(EP,BP) telemetry Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09cascade equilibrium solver solved: fb (forward-backward message passing) K=3 ↵Yuren Hao
beta=0.003 — gates 1.0000/0.9990/0.9946 across BP trajectory, L12 0.9999; trainer v2 single-sided EP readout + divergence guard (~5x BP) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09casc_eq_train: equilibrium-mode cascade-EP trainer (GS-reverse solver K ↵Yuren Hao
sweeps, two-phase ±β, EP readout at relaxed states) — the true-EP route C1 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
generated by cgit v1.2.3 (git 2.25.1) at 2026-08-18 17:38:15 +0000