diff options
| -rw-r--r-- | ONBOARDING.md | 5 | ||||
| -rw-r--r-- | docs/campaign/FINDINGS.md | 18 |
2 files changed, 21 insertions, 2 deletions
diff --git a/ONBOARDING.md b/ONBOARDING.md index 4361271..ce6e549 100644 --- a/ONBOARDING.md +++ b/ONBOARDING.md @@ -29,8 +29,9 @@ F(z) = −(z − x_in) + Attn(LN(z)) + FFN(LN(z)) − c·z ## 3. Where we are (results, C512, TinyStories-BPE) - **EP gradient ≈ exact BPTT gradient** (cosine ≈ 0.92–0.99 per component when the free phase is converged) — the learning rule is validated, not approximate. -- **Best val cross-entropy 1.8977** (warm-started, `ep_warm_fast`, still descending pre-lr-decay as of 2026-07-05; - previous era best 1.9313) vs a same-parameter BP transformer ~1.79; generates coherent children's stories. +- **Best val cross-entropy 1.7065** (warm-started `ep_warm_fast`, FINISHED 2026-07-07) — **beats the tuned depth-1 + BP reference (1.7921) by 0.086 and the parameterization-matched BP+EMA twin (1.9632, best of 8 configs) by 0.257**: + the equilibrium computation's iteration-depth dividend, measured. Generates coherent children's stories. - **From scratch: 2.0119** (`ep_self_restart`, a from-scratch lineage with restart-on-abort; finished 2026-07-05) — the old "~2.10 plateau" is broken, but the warm/scratch gap (~0.11) remains the crux (see §5). - The recipe **trains stably and approaches BP** at this scale. Model = C512 / H16 / T256, damped DEQ block. diff --git a/docs/campaign/FINDINGS.md b/docs/campaign/FINDINGS.md index e3fdcc8..6253527 100644 --- a/docs/campaign/FINDINGS.md +++ b/docs/campaign/FINDINGS.md @@ -671,3 +671,21 @@ Full write-up: **`SESSION_2026-06-24_HOPF_DIAGNOSIS_RESREG_FIX.md`**. This pins **Protocol (golden seeds are manufacturable).** Reg-light run → save every ≤100 steps → ARPACK dip-screen → warm-start from a certified dip (|λ|<1 at depth). `abl_delay` (queue arm0, --reg_delay 2000) is the fixed-switch approximation; if its switch lands mid-instability the refinement is **engage-on-dip** (res_ema<~2e-3 after step≥1200), not a longer delay. Corollary for the plateau: resreg-from-step-0 may prevent the edge-forged structures from ever forming (safe scratch 2.06–2.24 vs warm-from-dip 1.90). **Tooling verdicts.** `lead_rho` cold (≤40 iters) under-reads near-unity clusters by 0.02–0.06 and mis-classified s1000/s2400/s2800 as stable — **audits must use ARPACK**; lead_rho is for warm-started in-training tracking only. Hidden `jacreg==0` damping feedback found in the train loop (res-driven c up/down) — would have polluted single-reg ablation arms; new `--noadaptc` kills it (arms updated). + +--- + +## 2026-07-07 — FLAGSHIP FINISH: EP warm 1.7065 (beats every depth-1 BP reference); tolerance wave-1; PAR wall; governor first certifications + +- **`ep_warm_fast` DONE: best val CE 1.7065** (warm s2000, hr0.2, t2sel80, adaptive jr, compile). vs tuned BP anchor + 1.7921 (**+0.086**), vs parameterization-matched BP+EMA twin 1.9632, best of 8 configs (**+0.257** = the measured + iteration-depth dividend). From-scratch best remains 2.0119 (self_restart); the spectral-governor arms are the + live from-scratch recipe candidates (both dip-certified: s12 @1000 |λ|max 0.9999, s11 @1500 |λ|max 0.9951). +- **Tolerance wave-1 (warm s2000, 1500 steps, hr0.02):** STATIC imperfections free — wq8 2.6343 (better than base + 2.6618!), wq6 2.6578, wq4 2.7335 (mild), **wmis 1% 2.6588 (free — fabrication tolerance headline)**. DYNAMIC + per-pass force noise = cliff: fn 1e-3..3e-2 ALL pinned at ~3.195 (never improved past the first eval) — the + phase-independent-noise / 2r amplification exactly as the estimator theory predicts. Wave-2 (hr0.2 + lock-in) + running = the big-nudge rescue test. +- **The other wall (arXiv:2603.26969, Stern/Liu et al.):** non-reciprocal dynamics ⟹ exact physical gradient + impossible in principle (adjoint unavailable to forward physics) ⟹ our Wall-3 (J^T v) is a theorem, with a price + list (bidirectional crossbars / CET exemption / PAR bias / mixed-signal) — see HW_RESEARCH_FINDINGS 07-07 + addendum. PAR bias tax being measured at C512: runs/par_baseline (plain nudge, warm s2000) vs the corrected 1.7065. |