summaryrefslogtreecommitdiff
path: root/ep_run
AgeCommit message (Collapse)Author
2026-07-09cascade root cause found: tied readout + default N(0,1) embedding init = ↵Yuren Hao
pathological top-CE stiffness once predictions sharpen (sigma_tok~76 vs GPT-standard 0.02 giving ~1.6); add --untie and --tok_init; diag arms A/B/C Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09casc_eq_train v4: quality-governed estimator — periodic measured ↵Yuren Hao
cos(EP,BP) drives K/beta (spend when quality drops, relax when abundant); guards reduced to sanity-only Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09casc_eq_train v3: adaptive beta (tok-sigma stiffness compensation), ↵Yuren Hao
adaptive-K fb with contraction test, non-contraction bail, in-training cos(EP,BP) telemetry Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09cascade equilibrium solver solved: fb (forward-backward message passing) K=3 ↵Yuren Hao
beta=0.003 — gates 1.0000/0.9990/0.9946 across BP trajectory, L12 0.9999; trainer v2 single-sided EP readout + divergence guard (~5x BP) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09casc_eq_train: equilibrium-mode cascade-EP trainer (GS-reverse solver K ↵Yuren Hao
sweeps, two-phase ±β, EP readout at relaxed states) — the true-EP route C1 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09cascade: retire zil per user directive (EP route = equilibrium two-phase); ↵Yuren Hao
probe gains --init_sweep (state warm-start, EP-clean readout) and --sopt adam; B2 solver sweep Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09cascade: zil interleaved scheme = exact gradients at all depths (cos 1.0000 ↵Yuren Hao
L6-24, trajectory 0.9998+); casc_ep_train zil trainer; naive-relaxation depth failure documented Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09cascade ablation program: plan doc (5 claims, tiers 0-4), probe v2 ↵Yuren Hao
(jacobi/gsf/gsr schemes, full-theta gate, multi-batch), casc_bp_train ckpt producer Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09cascade_probe: layered-energy EP on L distinct standard blocks — free ↵Yuren Hao
equilibrium == plain forward (standard LLM inference), two-phase grad gate cos 0.9968 vs BP (L3 C128) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09lt_ep_train: --gov_keepjr now keeps the jr floor through reg_delay too ↵Yuren Hao
(fastfull arm: jr always-on, resreg post-warmup) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09backfill_wandb: replay historical run logs into W&B (ept-c512 / ept-tol / ↵Yuren Hao
ept-33m) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09lt_ep_train: --rr_floor (resreg floor during free/delay phase) for govfloor armsYuren Hao
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09lt_ep_train: optional W&B mirroring (--wandb/--wandb_run), failure-proof ↵Yuren Hao
logging of val/ema/best/res/jr/lr/rho/gov state Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-07SPECTRAL GOVERNOR (--governor): the from-scratch recipe candidate, ↵Yuren Hao
dip-map-calibrated Dip screening (4/4 seeds have dips; first-dip cluster 1200-1700, width 100-300 steps, depth |lam|~0.998 = s2000-class; s2 shows excursion->dip in sequence): reg-free until a scout (warm lead_rho, every gov_k=100, deep-400 subbatch) flags rho<0.985 after gov_min=1000, then ARPACK-certify all top-3 |lam|<1 -> leash ON (pair regs). Post-engagement: sustained rho_scout>1.02 x2 -> rescue boost (resreg x3 for 500 steps; the proven hr2 maneuver). abl_delay's fixed-2000 death explained: it engaged AFTER the dip cluster, mid-excursion. Two governor seeds launched on 107 (trained-state engagement is Pascal-safe per hr2 precedent). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-06dp_ep smoke PASSED: 2-GPU lockstep clean, ~107% linear scaling (allreduce ↵Yuren Hao
negligible vs seconds-long EP steps) val 6.0->5.83/100 steps at effective B48, res nominal, ranks never diverged. Speed package final ledger: pack 1.47x X DP ~linear (X optional AA-v2 1.3x). dip_screen.py: ARPACK screener for the dip-farm trajectories (running on freed 1080s, 2 chains). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-06Tier-3 gates: Anderson math-yes/impl-no (parked for v2); bf16polish UNSAFE ↵Yuren Hao
near-edge (eval-only); dp_ep.py ready Anderson: res 25-35x deeper per budget but 5.8x slower (naive history stacks + per-iter safeguard eval) — v2 = ring buffers + periodic safeguard, est +1.3x on the speed tier. bf16+20polish: 1.41x free phase, res parity, BUT z-diff 1.2e-3 — near-marginal operators contract too slowly for a 20-step polish (0.998^20≈0.96), same magnitude as the TF32 kill verdict and the estimator's 50%-sensitivity input. Predicted by our own depth/noise theory. Flags kept with warnings; neither ships for training. dp_ep.py: manual-allreduce EP DP (controller in lockstep, aligned collectives), smoke pending freed 1080s. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-06aggregate speed bench: speed tier hf+sd 1.47x; accuracy tier hf+sd+t80+avg ↵Yuren Hao
(0.91x, cos 0.94, avg free); compile demoted Full-ep_step wall times on quiet A6000 (warm s2000, B24), res parity across all 8 configs. compile only 1.12x at this shape (historical 1.46x was a different workload split); FULL cmp_sdpa saves 4% over eager at t80 — not worth the guard complexity. tforce_sdpa added (flash baked into compiled graph, flag-free so grad paths never see SDPA). bp_lm --tie probe in flight. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-06--holoavg shipped (gate 0.913->0.936 @t2sel160, batch2 +0.051) + farm wave-2 ↵Yuren Hao
tooling trend-aware stop + plateau averaging recovers the semi-convergence victims exactly as predicted. Estimator pack now: holofast+sdpa+t2sel80(+holoavg). Also: bp_lm stdinit/beta2/sched flags (anchor archaeology), dipfarm_freezer, staged tol_sweep.sh (gated on hr2 verdict), bp_sweep.sh. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-06bp_lm twin control + env forensics probes + holo_a_track_avg ↵Yuren Hao
(semi-convergence fix, ungated) BP+EMA 2x2 (local, healthy env): qknorm {1.9951, 1.9977}, no-qknorm {1.9728, 1.9732} -> parameterization-matched BP twin tops at ~1.97 vs EP 1.7888: the equilibrium computation's iteration/depth dividend = 0.18 CE from identical parameters. External anchor (tuned depth-1 BP, 1.7921): EP at parity. bp_lm on 107/2.3.1 gave 1.9823 ~= local -> plain backward exonerated on the pascal env; the 107 divergence (pair/floss/resreg dead by step 600) narrows to the EP reg/estimator loop. Single-step fingerprints all match (a/b/c/d) -> suspected intermittent kernel issue; discriminators in flight (torch-2.7 env probe + delay_hr2's step-2000 reg-on transition). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-05estimator forensics closed: cos ceiling = adjoint TRUNCATION + ↵Yuren Hao
SEMI-CONVERGENCE (not r, not anchor, not transients) Refutation chain: r-sweep flat (0.02-0.4); deep anchor (res 100x tighter) no gain; kappa brake monotonically harmful. t2sel window sweep at s2000: 40->80 lifts ALL batches (mean 0.889->0.936, truncation confirmed); past 80 SEMI-CONVERGENT (batch-dependent optimum; the inc-argmin t_best rule fails on rotating slow modes -> batch2 degrades 0.956->0.875 at 320). Early stopping IS the regularizer; iteration count = reg parameter. Shipping insight: holofast + sdpa + t2sel80 ~= old default wall-clock with cos 0.89->0.94. warm_fast (record) already ran t2sel=80 vs proven-scratch 40 — a real +0.05-cos hidden difference in the lineage table. Next lever: trend-aware stopping / t_best-neighborhood averaging. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-05r-sweep: cos(EP,BPTT) FLAT across r=0.02..0.4 on both operators — ↵Yuren Hao
nudge-SNR hypothesis refuted The 50% single-shot sensitivity is MULTIPLICATIVE (near-marginal T2 dynamics' state-sensitivity; signal and error scale with r together) — not additive noise divided by 2r. cos ceiling ~0.88 at near-edge states is set by T2 truncation + state sensitivity (0.98 at deeply-contracted states). hr-0.2's empirical wins are NOT estimator SNR. Hardware upside: algorithm indifferent to r across 20x -> nudge amplitude free to fight ADDITIVE readout noise. Surviving accuracy levers: kappa nbrake (Tikhonov, targets the real culprit), tail-window averaging over argmin selection. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-05--sdpa: fused flash attention in the no_grad relax loop — 1.45x free ↵Yuren Hao
phase, z* parity 4e-7 Scoped via blk._sdpa set only inside relax()'s loop (grad paths jvp/vjp/resreg keep the manual attention: no forward-mode-through-flash risk). Combined with --holofast: ~1.51x full-step exact-math tier. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-05holofast: exact halved-jvp AEP track — 1.55x nudged phase, gradient-gate ↵Yuren Hao
passed holo_a_track computed the doubled-batch jvp/vjp on [v0; -v0] at a shared anchor zbar — exact antisymmetric redundancy (phase deviations from the common mode are exact negatives). holo_a_track_fast computes at batch B and mirrors: single-eval parity 6e-7 (exact); trajectory-level 45% divergence SHARED with the original's own FD noise floor (1e-6 state noise -> 49% self-divergence — the 2r=0.04 finite difference amplifies fp noise; training averages it via pema/momentum). Ship gate: cos(EP,BPTT) orig vs fast indistinguishable (0.907/0.912, 0.853/0.853, 0.918/0.918). Timing 6.43->4.16s on the T2=40 nudged phase (contended GPU, relative). --holofast flag, default off; queued ablation arms deliberately stay on orig for fidelity. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-04queue v3: 5th arm abl_delay_hr2 (redx-faithful hr0.2 front) + every-100 ↵Yuren Hao
saves on delay arms for dip-screening Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-04magic-s2000 study: reg_delay/noadaptc flags, 4-arm queue v2, redx trajectory ↵Yuren Hao
audit - lt_ep_train: --reg_delay N (reg-free early phase: resreg/jr/floss/adaptc off for first N steps) + --noadaptc (kill hidden jacreg==0 damping feedback that would pollute single-reg ablation arms) - queue v2: 4 arms delay-first (abl_delay = reg-free 2k -> proven pair) - eig_traj/2/3: ARPACK audit of redx_traj — the run crossed the edge EARLY and oscillated (s1000 rotating-unstable, s1400 excursion mu=+2.1 self-recovered, s2000 the ONLY stable snapshot mu=-0.02, s2100/s2200 already back out) => s2000 is a post-excursion STABILITY-DIP capture, dip width <100 steps; learning survives mild instability (val fell through unstable stretches). lead_rho cold-40 under-reads clusters — NOT a classifier; ARPACK for audits. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-03floss-ept: graded finite-horizon LE penalty (--floss) + three-arm ↵Yuren Hao
from-scratch ablation queue - ep_step: floss block after resreg — unroll q=10 steps past z_T1 on a sub-batch WITH graph, rho_hat = mean per-step delta growth, one-sided relu(rho_hat - 0.995)^2, ramp keyed on (rho_hat - target) NOT resT1 (de-cliffed resreg: same fundamental path-LE quantity, linear early signal), capped at floss fraction of task-grad norm (resreg convention). - smoke: below-target = untouched (cos 1.0000); force-fire = finite grads, capped perturbation (cos 0.9803). - runs/abl3_queue.sh (runner live): waits for free GPU slots (0/1/3, GPU2 excluded), launches abl_floss (floss-only) / abl_resreg (resreg-only, never cleanly run) / abl_pair (proven 2.09 recipe, control) with identical remaining flags + seed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-03omega/norm-family refuted as stability signal; fingerprint story retracted; ↵Yuren Hao
eigreg v2 = true map-eigenvalue (spec_penalty) - eig_control: fix plain-PI bug (shifted PI for lambda_max of indefinite Sym); add lead_rho + spec_penalty (soft one-sided cap on |lam|(I+eps*J_F), 2-D Rayleigh-Ritz, matvec-only) — aep 'spectral' ported. eig_penalty demoted to diagnostic. - eig_recheck.py (Lanczos audit): omega=+5..+13 on ALL operators incl the stablest (s2000 +12.8 while true alpha=-0.02); gap omega-alpha~10; old 'warm -10.14 vs scratch +1.11' numbers were PI-mixture artifacts. RETRACTED. - eig_v2_smoke/depth: v2 mechanics validated vs ARPACK; z_T1 readings >1 are unconverged-state contamination (150: 1.009 -> 400/800: 0.997-0.999, mu=-0.02..-0.006 matching eig_probe); fixed-point top = BAND of slow modes. - lt_ep_train: --eigreg now spec_penalty (--eig_margin 0.995 = rho target); --fingerprint reports rho/Re_mu instead of num_abscissa. - ONBOARDING §4-7 + FINDINGS 2026-07-03: retraction + verdict (fundamental quantity = finite-horizon path LE / resreg axis; de-cliff via floss-ept; spec_penalty = measure-mode scalpel for a detaching Hopf pair). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-03Initial commit: ept — backprop-free equilibrium transformer (EP)Yuren Hao
Code (ep_run/), organized docs (docs/{method,campaign,hardware,outreach,paper}), analysis scripts (scripts/), ONBOARDING.md entry point. Large data/checkpoints git-ignored (share separately). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
generated by cgit v1.2.3 (git 2.25.1) at 2026-08-18 18:24:14 +0000