path: root/docs/method/METHODS.md

# Methods — EP-Trained Equilibrium Transformer Language Model

Complete technical notes for discussion: architecture, how attention/FFN are made EP-trainable,
the training rule, every stabilizer/regularizer and its reason, the LM setting, validation
methodology, results, and open problems. Code paths at the end. (Companion doc:
`/home/yurenh2/ept/FINDINGS.md` — the project arc and findings log.)

---

## 1. Problem statement

Train a transformer-class language model where **both attention and FFN learn without
backpropagation through the computation** — using Equilibrium Propagation: two (or N) relaxations
of the same dynamics plus a local contrast readout. The questions: (a) does it train at all,
(b) what does it cost vs the exact gradient (BPTT on the same architecture) and vs a standard
BP transformer at equal parameters, (c) what are the actual failure mechanisms.

Headline result (all at 14k steps, fully-controlled comparison): best EP model reaches
**val CE 1.676** (adaptive T1/T2, run R10). Like-for-like standard BP transformer (MLP=4 — the
same parameter shape as the thick block, see §2) reaches 1.610 ⇒ **total gap 0.066**, decomposed:
**architecture ≈ 0.025** (BPTT + the same stabilizer + same param-EMA on the identical block:
1.635) and **training rule ≈ 0.041** (EP 1.676 vs that control) — and EP beats the thin-matched
BP MLP=1 baseline (1.689). Unregularized BPTT *destabilizes* on long horizons (walks off the
contractive manifold, res→5e-2, best 2.021 — worse than its 3k run, 1.949): the stabilization
loop EP carries out of estimator necessity (residual-driven Jacobian-penalty controller) is what
the equilibrium architecture itself needs for long training. Random = ln 65 = 4.17.

## 2. LM setting (data, embeddings, readout, evaluation)

- **Corpus**: Shakespeare character-level (nanoGPT preprocessing): `train.bin`/`val.bin` uint16
  token streams + `meta.pkl`, vocab = 65 chars (~1.1 MB text). Local copy:
  `/tmp/lt_ep/data/shakespeare_char`.
- **Batching**: random crops, B=32 sequences × T=64 context; next-char targets (shift by 1).
- **Embeddings**: learned token table `tok ∈ R^{65×128}` (init N(0, 0.02²)) + learned absolute
  positional table `pos ∈ R^{64×128}`. Input injection `x_in = tok[idx] + pos`. **Embeddings are
  trained by EP too** — they enter the force through the input-clamp term −(z − x_in), so the same
  vector-field readout (Sec. 4) delivers their gradient. No pretrained embeddings, no BP path.
- **Readout head**: logits = z* Wh, `Wh ∈ R^{128×65}`, trained with its **own local CE gradient**
  ∂CE/∂Wh at the free equilibrium (loss-adjacent layer — local learning suffices; this is standard
  in EP setups and is not backprop through the dynamics).
- **Objective / eval**: mean next-token cross-entropy over all B·T positions; val CE = average over
  8 fresh validation batches, computed by running the same free-phase relaxation (T1 steps) used in
  training — i.e. the eval graph equals the inference graph. Random baseline ln(65) = 4.174.
- **Model size**: C=128, H=4 heads, single equilibrium block (weight-tied recurrence ⇒ effective
  depth = T1). Parameter matching to the BP baseline depends on the variant: the **thin** block's
  Hopfield memory (Wm: 128×256 ≈ 33k) matches BP **MLP=1** (2C² ≈ 33k); the **thick** block's
  untied FFN (fc+pj = 2·4C² ≈ 131k) matches BP **MLP=4** (131k) — so thick-block results compare
  against MLP=4 (1.610), thin-block results against MLP=1 (1.689).

## 3. Architecture: the equilibrium block

State `z ∈ R^{B×T×C}` (one vector per token position). Dynamics ż = F(z); inference = relax to
fixed point z* (Euler: z ← z + ε F(z), ε=0.1, T1=150 steps), predict from z*.

We built four force variants (all share the input clamp and the readout):

### 3.1 `thick` — DEQ-transformer block (the winner)

```
F(z) = −(z − x_in)                            input clamp (leak toward embedding)
     + Attn(LN1(z))                           causal multi-head softmax attention, separate WQ WK WV WO
     + W2 · GELU(W1 · LN2(z) + b1) + b2       untied 4× FFN (W1: C→4C, W2: 4C→C)
     − c·z                                    damping (c=1–2)
```

LN1/LN2 carry learned affine (g, b). This is exactly a pre-LN transformer block written as a
*force* instead of a layer stack — same form DEQ uses as its fixed-point map. It is strongly
**non-conservative**: no scalar energy has this gradient (Q≠K asymmetric coupling, untied FFN).
EP is made exact for it via the AEP correction (Sec. 4.2).

### 3.2 `real`/thin — Hopfield-FFN + damped real attention

```
F(z) = −∇_z [ ½‖z − x_in‖² + E_mem(z) ] + s·(Attn(z) − c·z)
E_mem(z) = −Σ relu(z Wm)²                     modern-Hopfield / dense associative memory, Wm: C×256
```

The FFN here is **energy-based**: E_mem is the dense-associative-memory energy; its force
2·relu(zWm)Wmᵀ is a one-hidden-layer FFN with *tied* weights (Wm in, Wmᵀ out). Attention remains a
raw non-conservative force with damping. This was our first stable trainable variant.

### 3.3 `energy` — fully conservative attention (CET-style)

Attention folded into the energy: `E_att = −(1/γ) Σ_heads,i LSE_j(γ q_i·k_j)` (causal-masked,
**tied value** — the force of this energy mixes values v=k), plus `½c‖z‖²` confinement because
E_att+E_mem alone are unbounded below. F = −∇E exactly ⇒ classic EP applies with no correction.
This is the CET (Høier/Kerjan/Scellier) route, which we reproduced separately on vision
(masked CelebA/CIFAR completion, EP ≈ TBPTE). Trade-off: tied value + reciprocal coupling = the
least expressive attention.

### 3.4 `mono` — monDEQ-structured contraction

`F(z) = −(m·z + z PᵀP) + z(Q−Qᵀ)ᵀ + x_in − ∇E_mem + s·Attn(z)` — the linear part is a monotone
operator by construction (Winston–Kolter): symmetric part ⪯ −m·I guaranteed, antisymmetric part
(Q−Qᵀ) free (non-reciprocal coupling at no stability cost). Guaranteed unique fixed point for the
linear core; softmax attention sits on top with gain s. Ablation for "how much does guaranteed
contraction cost": BPTT-mono = 2.11 vs BPTT-thick 1.95.

### How attention is "EP-ified": the two routes, explicitly

1. **Energy route** (3.3): make attention conservative (tied value, LSE energy) so F = −∇E and
   vanilla EP is valid. Cost: expressivity (Q≠K asymmetry and free value are what make attention
   attention).
2. **Force route** (3.1, 3.2 — ours): keep real attention as a non-conservative force and repair
   the *estimator* instead, with the AEP correction (Sec 4.2) which restores exact gradients for
   non-reciprocal couplings. Validated gradient cosine vs autograd: attention 0.99, FFN 1.00,
   full LM block 0.99 (and FA, for contrast, gives Q/K/V ≈ 0.25, upstream FFN ≈ −0.01).

### How attention and FFN are trained *jointly*

There is no per-module schedule or pipeline: attention, FFN, LN affines, and embeddings are all
terms of the **same force** F. One free relaxation + one nudged ensemble produce the contrast state
`a`; every parameter θ gets its gradient from the single vector-field formula ∇θ = ∂⟨a, F(z*,θ)⟩/∂θ
(Sec 4.1), which decomposes into purely **local** per-term updates (each force term touches only its
own parameters). The readout Wh is the only separately-trained parameter (local CE gradient).

## 4. Training rule

### 4.1 Vector-field (force-form) EP with symmetric nudging

- **Free phase**: z⁰ = x_in; z^{t+1} = z^t + ε F(z^t), T1=150, ε=0.1 → z*. Monitor relative
  residual `res = ‖z⁺−z*‖/‖z*‖` (one extra step) — the load-bearing health signal (Sec 5).
- **Nudged phases** (±β, β=0.02, T2=20 steps from z*): relax the augmented force
  `F(z) ∓ β ∇_z CE(z)` (the CE gradient w.r.t. the *state* — local at the readout).
- **Contrast**: `a = (z₋ − z₊)/(2β)` ≈ −dz*/dβ (centered ⇒ O(β²) bias; Laborieux-style symmetric
  nudging).
- **Parameter update** (force form, valid for non-gradient dynamics): for all force params θ,
  `∇θ L ≈ ∂/∂θ ⟨a, F(z*; θ)⟩` — one autograd call **at the fixed point only** (this is a local
  Hebbian-style contrast in θ for each force term; autograd here is per-term bookkeeping, not
  backprop through time/steps).
- At a converged fixed point and β→0 this equals the implicit/equilibrium gradient
  (Scellier–Bengio; Ernoult: EP ≡ BPTT stepwise under convergence).

### 4.2 AEP correction (non-conservative repair)

For non-conservative F the naive nudged phase linearizes around z* with Jacobian J, but the correct
adjoint needs Jᵀ. Following **AsymEP** (Scurria, Vanden Abeele, Mognetti, Massar, "EP for
Non-Conservative Systems", arXiv:2602.03670), we add to the nudged force the term `−(Jv − Jᵀv)`,
v = z − z*, where J = ∂F_nc/∂z at z* and F_nc = the non-conservative part (attention, or
attention+FFN for `thick`). This is **identical to their `−2 A_J(z*)(z−z*)`** (A_J = antisym part of
J; `Jv−Jᵀv = (J−Jᵀ)v = 2 A_J v`) — **the correction is theirs, not ours**. What is ours on this
line: (i) the **matrix-free jvp+vjp** form (one of each per nudged step) — AsymEP builds the full
Jacobian explicitly and decomposes it, which is infeasible at transformer state dim B·T·C; (ii) the
application to **softmax attention** (data-dependent Jacobian — they test only feedforward
nets/Hopfield on static MNIST/CIFAR, no attention/sequence); (iii) the **holomorphic combination**
(§4.3 — the correction is real-linear so it preserves holomorphy; they use plain ±β); (iv) the
**common-mode-tracking** linearization variant (§4.3/below). Their force-form **VF** readout (= our
`⟨a,∂F/∂θ⟩`) is *prior art* and is the baseline that collapses without the correction (their CIFAR
VF = 10% chance), matching our cos≈0.25 for uncorrected attention. Effect: nudged-phase Jacobian J → Jᵀ ⇒ a approximates the *adjoint*
response −(I−Jᵀ)⁻¹-type solve ⇒ exact gradients for Q≠K attention (measured 0.99–1.0).
Caveats: the correction is **linearized at z***, so the nudged trajectory must stay in the
linear-response window — T2≈20 at ε=0.1 is inside; T2=60+ can leave it (Sec 7).

### 4.3 Holomorphic EP upgrade (current)

Replace the 2-point real ±β difference by N points on a **complex circle** β_k = r·e^{2πik/N}
(Laborieux–Zenke 2022): relax the *holomorphically extended* dynamics (manual complex LN with
non-conjugate variance, softmax as exp-ratio, tanh-form GELU; the AEP correction is linear with
real coefficients so it preserves holomorphy — apply to Re/Im parts separately), then read
`a = −Re[(1/Nr) Σ_k e^{−iφ_k}(z_k − z*)]` (discrete Cauchy formula; bias O(r^N) instead of O(r²)).
**No clamps inside the holomorphic nudge** (clamps are non-analytic and break the bias order).

Probe findings (cos vs long-horizon-BPTT reference, 300-step-pretrained thick block):
- The **clamps were the dominant estimator error at marginal residuals**: at res 1.6e-3, plain EP
  cos 0.27 → clamp-free 0.89. (The clamps existed to protect early training; they were silently
  poisoning mid-training updates whenever res drifted up.)
- N and r are flat (N=2…8, r=0.02…0.2 all ≈ equal): finite-β bias and 1/β noise are *not* the
  binding error at this scale.
- The remaining ~0.12 misalignment is **T2 truncation**: with stable nudged dynamics, T2=120 →
  cos 0.985. But on slow-mixing batches long T2 diverges (AEP linearization error compounds on
  near-critical modes). Step-size-based early stopping FAILS (non-normal transient growth triggers
  it at t≈6–39; same pathology that makes spectral radius the wrong free-phase signal).
- **Adaptive T2, solved by hindsight selection** (`holo_a_select`): run the nudged phases to
  T2max=120 in lockstep, snapshot the contrast a_t every 10 steps, return the snapshot with the
  smallest increment (= most settled), early-exit only on clear blowup (inc > 5× the running min).
  Judging stability by increments of the *quantity of interest* — not step sizes — makes transient
  growth harmless. Probe: never worse than fixed T2=20; mean cos 0.871 → **0.932** at tight
  equilibria (0.853 / 0.987 / 0.956 per batch; the dangerous batch self-limits to t≈20–30).
- **Adaptive T1 companion**: long-T2 gains require a tight free phase (res ≲ 1e-4; at res ~1e-3
  long T2 actively hurts). So the free phase is two-stage: the λ-controller's residual signal is
  still sampled at T1=150 (R9 semantics preserved — no new λ war), then relaxation continues in
  chunks of 50 until res ≤ 1e-4 (cap 500) before the nudged phases. Compute buys tightness;
  λ pressure does not.

Training outcomes: **R7** (N=2, r=0.02, fixed T2=20): best 2.0289, faster wall-clock than plain EP
(3.3 vs 2.45 it/s — the holomorphic nudge's ∇_z CE is closed-form). **R10** (R9's controller +
adaptive T1/T2): **best 1.6755** (sustained EMA plateau 1.68–1.70 around step 8–10k; ~0.7 it/s).

## 5. Stabilization & regularization — what, and exactly why

**The governing fact (measured, not assumed):** the EP estimator has a **validity threshold** in
free-phase residual. Gradient cosine vs exact reference: res ≈ 5e-5 → 0.85–0.88; res ≈ 1e-3 →
0.2–0.9 (batch-dependent); res ≈ 3e-3 → ≈ 0–0.5; res ≈ 1e-2 → noise. BPTT has no such threshold
(it differentiates the actual finite computation, converged or not) — *this asymmetry, and nothing
deeper, is the EP-specific difficulty*. There is **no structural ceiling**: an early "EP caps at
~2.5" verdict was refuted (it conflated two undertrained/invalid-regime runs; see FINDINGS).

Each stabilizer and its reason:

1. **Damping −c·z** — creates/strengthens a fixed point for raw attention forces; *required* for
   the thin/real variant (attention alone has no fixed point at high gain: residual floor ~3e-2,
   no equilibrium to find). **Caveat for LN-inside blocks (`thick`)**: damping shrinks ‖z*‖ and the
   LN Jacobian scales like 1/σ(z) ⇒ damping *inflates* the effective Jacobian — measured: thick
   plain-relax residual 8.8e-3 at c=0 vs 3.4e-2 at c=2. So for `thick`, c is kept small (1) and is
   NOT the stabilizer; the Jacobian penalty is.
2. **Soft Jacobian penalty** λ‖J_nc(z*)‖²_F (Hutchinson estimator: one jvp on a random probe vector,
   differentiated w.r.t. θ; Bai et al. 2021 "Stabilizing Equilibrium Models by Jacobian
   Regularization") — the actual stabilizer: keeps the free phase contractive ⇒ keeps the estimator
   inside its validity region. Soft penalty ≻ hard constraints (spectral-norm capping the attention
   matrices to ρ=0.9 was tried: too restrictive, kills learning — consistent with FRE-RNN-style
   regulation being preferable to hard projection).
3. **Why the control signal is the residual, NOT the spectral radius**: the attention/block Jacobian
   is highly **non-normal** — transient growth is invisible to eigenvalues (measured: ρ(J)=0.94
   "stable" while the relaxation diverged at res 0.21). The one-step residual *is* the transient;
   control on it.
4. **Continuous λ controller**: λ ← clip( λ · (res_EMA/target)^0.3 , floor, 16 ), per step.
   - **EMA on the signal (0.9)**: the raw residual is noisy; a multiplicative controller on a noisy
     signal random-walks (measured thrash λ 0.5↔13 when the target sat at the noise floor) and the
     thrashing λ itself perturbs training. EMA removed it and gave the current best run.
   - **Target = 5e-4**: just inside the validity threshold (few·1e-4), with margin; NOT tighter —
     demanding res ≪ threshold buys nothing and costs expressivity (a 2e-4-target run was worse).
     For reference, BPTT's own optima sit at res ~1e-3–2e-2: good solutions are only mildly
     contractive; we ask for slightly more than BPTT needs, because our *estimator* needs it.
   - **Floor**: λ may shrink when res is healthy but **must not vanish — the floor is
     load-bearing**. Floor=λ₀ (never off) is a permanent tax (2.150); floor 0.1 is the sweet spot
     (2.078 → 2.047 with signal-EMA → 2.029 with the holomorphic estimator). Two independent runs
     prove λ≲0.02 is fatal at *any* stage: λ→0 from the start (R2) and λ-floor annealed with lr
     (R6) both ended in the same death — val CE 60–77 with res ≡ 0.0. Post-mortem: this is **an
     explosion disguised as convergence by floating point**, not a dead state: ‖z*‖ and the
     uncapped parameters (tok/pos/fc/pj) blow up under temporarily-invalid gradients until
     ε·F < ulp(z) and the relaxation freezes (res = 0 by absorption), with huge confidently-wrong
     logits. The λ penalty (whose θ-gradient touches fc/pj/LN/attention) is what keeps that basin
     out of reach; it cannot be annealed away. The late-drift hypothesis "persistent penalty
     gradient vs vanishing task signal" is hence only half-true — the persistent pressure is also
     the anti-collapse mechanism. Current anti-drift attempt: **parameter EMA** (decay 0.999,
     evaluated alongside raw weights), which targets late-phase estimator-noise wander without
     touching the stability loop at all.
5. **Weight-norm caps** (renorm to 3× init norm on WQ,WK,WV,WO,Wm,Wh after each step): blunt safety
   net against runaway during transients when the estimator is temporarily invalid. Rarely binding
   in healthy runs.
6. **Nudge clamp g.clamp(±2) and AEP-correction clip (‖corr‖ ≤ ‖F‖)** — *legacy*: protected the
   nudged relaxation early in training, but measured to be the main estimator error at marginal
   residuals (cos 0.27 → 0.89 once removed). Replaced by the clamp-free holomorphic nudge; a
   non-finite-gradient step-skip + the λ controller now carry the early-training safety.
7. **Optimizer**: AdamW lr 1e-3, wd 1e-4, cosine to 5%, grad-norm clip 5.0, skip non-finite steps.
   β(EP)=0.02, ε=0.1, T1=150, T2=20 everywhere unless stated.

### 5.x T1-residual penalty (`--resreg`) — defend the evaluated state (2026-06-20)

EP's gradient is the fixed-point/implicit gradient: it only cares WHERE the fixed point is, not how fast the
relaxation reaches it, so it has no reward for keeping the block contractive. BPTT — differentiating the finite
T1=150 unroll, which is what eval actually uses — gets that reward implicitly (a non-converging unroll → bad
output → high CE). This asymmetry is why frozen-jr EP diverges past ~2.09 (res inflates → forward bifurcates to
a limit cycle) while exact-BPTT with the identical recipe descends to 1.72 (see FINDINGS 2026-06-20; the EP run
refines the free phase to t1max=300=z* and grades there, so it never feels the residual of the evaluated z150).

The fix gives EP that missing term explicitly — penalize the T1 free-phase residual of the state actually
evaluated, `z150 = relax(xin, T1)` taken BEFORE any t1max refinement:
- `R_res = ‖ε·F(z150)‖² / (‖z150‖²+ε)`, gradient w.r.t. θ with z150 detached (`blk.tforce`);
- scaled task-relative: `ratio = resreg·min(1, res@T1 / 2e-2)`, deadband `res@T1 > 7e-4`,
  `λ = ratio·‖g_task‖/‖g_res‖`, added to the EP gradient.
- **Run with `res_gate=0`** — the validity gate early-returns (jacreg-only) above the gate, which would bypass
  the penalty exactly when res is high. Keep `t1max=300` (estimator accuracy) + the penalty (defends z150).

Analog-compatible (one extra force measurement + the same local vector-field gradient, no digital root-finder)
and more targeted than jacreg (which penalizes ‖J_nc‖_F, not the actual residual vector that explodes). Validated
res-tight through step 1000 / best 2.0573 (past the 2.09 wall) before a /tmp wipe; full re-validation pending.

## 6. Validation methodology (how we know the estimator/claims are right)

- **Gradient-cosine probes**: at a fixed realistic operating point (300 BPTT steps from init —
  no contraction penalty, the "natural" weight region), compare every estimator against a
  long-horizon BPTT reference (T1=400), per parameter group (attn / ffn / LN / emb). This is what
  exposed the validity threshold, the clamp damage, and the T2 truncation.
- **Horizon control**: BPTT-150 vs BPTT-400 cosine is itself only 0.35–0.77 on slow-mixing batches
  — the "finite horizon vs true equilibrium gradient" cost is shared by everyone, EP is not
  special; at matched horizon EP is within ~0.15 of BPTT.
- **BPTT-as-ablation**: BPTT on the identical architecture isolates *training-rule* cost (EP−BPTT)
  from *architecture* cost (BPTT−BP). BPTT is an ablation, not the target; BP is the target.
- **Same-graph eval**: val CE is computed through the same T1-step relaxation used in training, so
  no train/eval mismatch flatters either method.
- **Gradient-cosine has a lifecycle**: early/mid training it measures estimator quality (0.93 →
  0.79–0.85 across scale); late in training, at slow-mixing trained points, even two *exact*
  gradients at different horizons decorrelate (cos(BPTT-150, BPTT-800) = 0.25 at the trained S1
  point) — no single "true gradient" exists to cosine against, and the meaningful arbiter becomes
  the training outcome on the horizon-matched eval objective. Validity-threshold claims here are
  early/mid-phase statements. Late-phase corollary: EP's res target 1.5e-3 at S1 is already
  optimal — cos rises monotonically with tightness and the loose-weights/refined-measurement
  variant nulls at training level: the measurability-contraction tax is rigid across the interval
  (the physical escape is oscillatory/lock-in measurement, not operating-point engineering).

## 7. Results

**14k-step matched comparison (the honest table; thick block ≈ BP MLP=4 in parameter shape):**

| training rule | architecture / recipe | best val CE |
|---|---|---|
| BP | standard transformer, MLP=4 (**like-for-like for thick**) | **1.610** |
| BPTT + R9's λ-controller + param-EMA | thick (exact grad, same stabilization as EP) | **1.635** — tail stable |
| **EP (R10)** | **thick; R9 + adaptive T1 (refine to res≤1e-4) + adaptive T2 (selection, cap 120)** | **1.676** (EMA plateau 1.68–1.70) |
| BP | standard transformer, MLP=1 (thin-matched) | 1.689 |
| EP (R9) | thick; holo nudge + recalibrated controller (target 1.5e-3, λmax 4) + param-EMA | 1.740 |
| BPTT (exact grad) | thick, unregularized | 2.021 — **destabilizes late** (res→4.7e-2, val→3.0) |

3k-era and ablation numbers (shorter schedule):

| run | best val CE |
|---|---|
| BPTT thick, 3k (its best showing) | 1.949 |
| EP R7: holo estimator, old tight controller (target 5e-4) | 2.029 (late λ pinned 16 ⇒ drift) |
| EP R8: R7 + param-EMA | 2.031 (EMA alone ≠ fix; the λ fight dominates) |
| EP R5/R3: plain estimator generations | 2.047 / 2.078 |
| BPTT monDEQ / thin, 3k | 2.111 / 2.206 |
| EP R2 (λ→0) / R6 (λ-floor∝lr) | 2.357 / 2.501 — both die by fp-absorption explosion |
| random | 4.174 |

Reading: (a) final decomposition — **architecture tax ≈ 0.025** (1.635 vs 1.610), **EP rule tax
≈ 0.041** (1.676 vs 1.635), total **0.066** to the like-for-like BP transformer; EP beats the
thin-matched MLP=1 baseline. (b) EP beats *bare* BPTT at both horizons, but the controlled
comparison shows most of that win was EP's mandatory stabilization loop doubling as regularization
— bare exact-gradient training walks off the contractive manifold at 14k, and the same controller
that EP cannot live without lifts BPTT to 1.635 (also beating MLP=1): **the contraction controller
is good for the architecture regardless of training rule; EP simply forced its discovery.**
(c) The estimator and the controller must be **co-designed**: upgrading the estimator
(holomorphic, clamp-free) widened the validity region from res≲5e-4 to ~1.5e-3, and re-calibrating
the controller to that wider region (R7→R9) was worth **0.29**; adaptive T1/T2 (R9→R10) was worth
another **0.064**, matching the probe's cos 0.871→0.932. (d) **Multi-seed confirmation (3 seeds per arm)**: EP
1.6755/1.6851/1.6786 → **1.680 ± 0.005** vs BPTT+controller 1.6348/1.6459/1.6365 →
**1.639 ± 0.006**; the rule tax is **0.041 ± 0.005 (~9σ)** — real, tightly reproducible, and
consistent with the measured estimator misalignment (cos 0.85–0.93).

**Scale rung S1 (TinyStories char, C=256 H=8 T=256, 0.92M params; random ln127 = 4.84):**

| run | best char-CE (BPC) |
|---|---|
| BP same-shape, 14k | 0.827 (1.19) |
| **BPTT-ctl, loose target 1e-2, 14k** | **1.009 (1.46)** |
| BPTT-ctl, tight target 1.5e-3, 14k | 1.521 — ⇒ **controller-mismatch tax 0.51** |
| EP v4b (validity gate, lr 1e-3, 20k from scratch) | 1.393 |
| EP L2 (v4b recipe, 40k from scratch) | 1.214 |
| **EP warm-track (v4b → phase-2: common-mode tracking + loosened target)** | **1.141** — EP champion |

S1 scale lessons: (a) containment must scale with model size (λ ceiling, cap list, fuse);
(b) the **validity gate is load-bearing** — off-equilibrium EP updates poison weights (three
deaths before the gate, alive after); (c) the estimator validity threshold tightens with scale
(res 1e-4 → 1e-5 for full quality; rescue is compute-bounded, saturating at cos ≈ 0.85);
(d) **the controller operating point is part of the training rule**: EP needs validity-tight
targets, exact-gradient methods want loose ones — match controllers for rule-tax measurements,
but report each method at its own best operating point for ceilings.

**Optimizer pricing (S0-Shakespeare, R10 recipe, 8k steps; AdamW ≈ 1.70 at 8k):** EP-SaI
(per-tensor lr from init g-SNR, frozen = one calibrated gain line per array) **2.048**;
SGDM 2.166; Lion 2.175; Lion+LARS 2.244. Per-tensor calibration recovers ~0.12 of the 0.47
uniform-scale gap; the remaining ~0.35 measures the value of per-coordinate adaptivity under EP's
noisy heteroscedastic gradients. Pretraining therefore stays in the digital shell; fine-tuning is
exempt (SGD suffices in the RL/fine-tune regime, with <0.02% sparse updates — an endurance gift;
Mukherjee et al., arXiv:2602.07729).

**Hardware twin v4 (S0, 8-bit program-verify + 30% static mismatch + σ=1e-4 white + 4× restart
averaging): best 1.937 — 90% of the clean improvement** (clean 1.68). Noise laws measured:
contrast pollution strictly linear in σ; √N restart averaging; snapshot SNR ≈ 1/53 at σ=0.3%,
r=0.2 ⇒ hardware needs ~10³–10⁴ lock-in averages per update (ms at MHz loops — physically trivial,
digitally prohibitive: the noise dimension repeats the compute story). Discovery en route: the
frozen AEP correction has a clean-environment instability at nudge horizons ≳150–300 steps
(spectrum of J(z) − J* + J*ᵀ uncontrolled) — windows ≤120 steps + restart averaging circumvent it;
the single-trajectory oscillatory (true lock-in) estimator awaits a fix for this horizon limit.

## 8. Open problems

1. **Late drift — mostly solved, mechanism identified**: the drift was the controller *fighting*
   the weights (λ pinned at max enforcing a target the upgraded estimator no longer needs; R7/R8
   tails). Re-calibrating target/λmax to the holomorphic estimator's wider validity region removed
   the fight (R9: λ stays 0.1–0.5, tail drift shrank from ~0.3 to ~0.15 above best). Refuted
   routes: λ-floor annealing (R6 ⇒ fp-absorption explosion — the floor is load-bearing);
   param-EMA alone (R8 — smooths wobble ~0.05, can't fix the fight). Residual ~0.15 tail drift
   remains open (estimator direction bias near optimum is the suspect).
2. **Adaptive T2 — SOLVED** by hindsight snapshot selection (§4.3): judge by increments of the
   contrast estimate, not step sizes; select the most-settled snapshot. Probe mean cos 0.932;
   training −0.064 val CE (R10). Possible refinements: larger N with selection, per-batch T2max.
3. **Mixing time**: slow-mixing equilibria make *all* gradients horizon-expensive (BPTT included);
   conditioning the dynamics for fast mixing (preconditioned/Anderson relaxation that preserves the
   EP contrast structure) is unexplored here.
4. **Scale**: depth-1 block, char-level, C=128. The mechanisms (validity threshold, non-normality,
   controller design) are dynamics-level and should transfer; the constants will move.

## 9. Hardware translation — can this now-complex algorithm still run on EP hardware?

Audit of every component of the final recipe (R10) against an analog/in-memory substrate. The
surprise: most of the added "complexity" is *control*, and control is cheap in analog; several of
our fixes specifically REMOVED digital artifacts.

| algorithm component | analog realization | difficulty |
|---|---|---|
| free phase (T1≈500 Euler steps — our digital bottleneck) | physical settling, ns–µs, "free" | trivial (hardware's whole pitch) |
| adaptive T1 ("relax until res≤1e-4") | settling detector = comparator on dz/dt | trivial |
| symmetric nudging ±β | output-node current injection | standard EP hardware |
| **holomorphic N-point circle** | **AC-modulated nudge + lock-in (homodyne) detection** — the Cauchy sum over phases IS lock-in readout; this is Laborieux–Zenke's "finite-size oscillations" taken literally | standard measurement technique; *more* native than DC differencing, and the standard weapon against analog noise floors |
| clamp removal (our biggest estimator fix) | hardware never had clamps; saturation is smooth | already done by physics |
| VF update ⟨a, ∂F/∂θ⟩ | local Hebbian outer product (contrast × presynaptic activity) per crossbar; autograd was only digital bookkeeping | native |
| λ-controller (residual → EMA → multiplicative λ, floor/cap) | a **neuromodulator/homeostatic loop**: 1 measurable scalar (settling ripple) → RC filter (the EMA) → log-domain integrator with rails (floor/cap) → global broadcast scaling a local anti-Hebbian (contraction) rule | a handful of analog components |
| λ floor = anti-collapse (R2/R6 lesson) | minimum leak conductance — never let homeostasis switch off | natural |
| adaptive T2 (snapshot selection) | sample-and-hold bank on the *contrast* signal + stability gate on the OUTPUT quantity (the transferable lesson: never gate on state velocity — non-normal transients fool it) | cheap |
| weight caps (3× init) | device conductance range | physics gives it free |
| param-EMA | slow/fast weight pairs (volatile + nonvolatile device) | known proposals |
| AdamW | per-synapse capacitor for momentum; second moment is the real gap (shared by all analog-training schemes) | open engineering |
| softmax attention + LN circuits | analog WTA / divisive-normalization primitives; data-dependent T×T attention in-memory | hard, but an *inference*-hardware problem shared by all analog-transformer efforts, not EP-specific |

**The two genuine research obstacles:**

1. **The AEP correction (J → Jᵀ) in physics.** Crossbars give Wᵀ for free (drive the other side),
   but the correction needs the *circuit's* transposed Jacobian, including data-dependent softmax
   parts. The classical answer exists: the **adjoint network** (Director–Rohrer 1969, circuit
   sensitivity theory) — constructible by reversing non-reciprocal elements; nobody has built
   AEP-learning with one yet. The alternative is a measured price list from our own ablations:
   accept *reciprocal* (energy-based, tied-value) attention and skip the correction entirely,
   costing ~0.15–0.2 CE (monDEQ 2.11 / energy-mode vs thick 1.95 under exact gradients).
2. **Precision budget vs the validity threshold.** Analog noise floors (~1%, 8–10 effective bits)
   sit exactly where we measured EP gradients dying (res ~1e-2). Mitigations, all quantified here:
   the N-point estimator tolerates **r=0.2** (10× nudge signal at equal bias — probe: flat in r);
   lock-in detection buys orders of magnitude of SNR below the noise floor; free and nudged phases
   run on the *same* devices so static mismatch cancels in the contrast (EP's structural advantage
   over on-chip backprop — note the adjoint path partially forfeits this and needs care); and
   hardware's 10⁶× speed headroom converts to phase-averaging. Our cos-vs-residual and cos-vs-r
   tables (§5, §4.3) are, read this way, the **spec sheet** for an analog EP-transformer design.

**Memristor-crossbar specifics — the Jᵀ question.** Needing Jᵀ does NOT disqualify memristor EP
platforms; crossbars are the most transpose-friendly analog substrate there is (drive rows→read
columns = Wx; drive columns→read rows = Wᵀy — the property on-chip-BP designs rely on). The fork:
(a) *passive-reciprocal* platforms (resistor-coupled, Kirchhoff/coupled-learning style) are J=Jᵀ by
physics — no correction needed, but they cannot express non-reciprocal attention even at
inference; on these, run the **reciprocal recipe**: LSE energy attention (tied value) + Hopfield
FFN — whose force 2·relu(zWm)Wmᵀ uses ONE crossbar driven in both directions — at the measured
~0.15–0.2 CE expressivity cost, zero hardware changes. (b) *active-periphery* platforms
(DAC→crossbar→ADC loops; periphery already breaks reciprocity) get Jᵀ as **transposed reads of the
same arrays + frozen local gains** (gelu′, softmax p, LN 1/σ from the free-phase operating point)
— the Director–Rohrer adjoint network in crossbar form: a periphery/routing redesign, not a
different device technology. Scope note: Jᵀ appears ONLY in the nudged phase applied to one vector;
the free phase needs nothing, weight updates are local outer products, and even the λ penalty needs
only ‖Jv‖² (forward perturbation + response energy, no transpose). Unlike on-chip BP there is no
per-layer activation storage or strict reverse scheduling — one held operating point z* suffices.
Numerics for the hardware team: large nudge amplitude r≈0.2 + multi-phase/AC (lock-in) readout is
validated equivalent to r=0.02 (10× signal headroom); small-signal DC differencing dies at ~1e-3
noise (our tf32 experiment: cos→−0.03). Suggested collaboration phasing: (1) reciprocal demo on
the existing rig (zero redesign, pay 0.2), (2) transposed-periphery nudge → full AEP, buy it back.

**On "training arbitrary analog circuits"** (the bigger question): classic EP requires
energy-based (reciprocal) circuits. AEP lifts this to *any circuit with a stable fixed point* —
IF the antisymmetric correction is realizable (adjoint network) or waived (reciprocal trade).
What this project adds to that picture is the missing stability half: **training pushes arbitrary
circuits off the contractive manifold** (bare-BPTT-14k showed even exact gradients walk off it),
and a residual-driven homeostatic controller both prevents this AND improves learning — with the
hard constraint that its floor never anneals to zero (fp-absorption collapse; in hardware:
latch-up). Combined with agnostic/physical EP (Scellier et al. 2022 — no circuit model needed,
contrast is measured) and small-scale physical demonstrations (Dillavou et al.'s self-learning
resistor networks; Laydevant et al.'s Ising-machine EP, 2024; memristor activity-difference
training), the pieces for "arbitrary stable analog circuit + adjoint or reciprocity + homeostatic
contraction control = trainable" are all individually demonstrated; this work supplies the
control law and the quantitative budgets.

Inference note: causal attention's lower-triangular coupling means autoregressive generation
settles *incrementally* — a new token's state relaxes with past states frozen, so EP inference is
one physical settling per token, not a re-relaxation of the sequence.

**Component BOM (assuming the current recipe survives the ladder unchanged):**
(1) bidirectional-read analog weight arrays (ReRAM/PCM/analog-Flash/gain-cell/switched-cap) — all
W and Wᵀ including the AEP adjoint reads; (2) state-integrator arrays (capacitor+OTA per state
variable; K·T·C nodes — ~1M at the 33M demo, ~17M at 0.6B; the sequence dimension T dominates);
(3) analog attention primitives — large-fan-in current/charge-domain softmax-WTA + T² score
sample-and-hold for the frozen nudge gains — **the hardest, least-shelf-ready component**;
(4) divisive-normalization circuits (LN/RMS/qk-norm); (5) mixed-signal periphery: DAC/ADC arrays,
S&H banks, and **lock-in (synchronous-detection) channels** — large-r AC nudging is mandatory
(small-signal DC differencing dies at analog noise floors; measured digitally via the tf32
experiment); (6) control plane: settling comparator + RC filter (res-EMA) + log-domain integrator
with rails (λ controller) + a global **learn-enable line (= the validity gate)** + fuse — a
handful of components or one MCU; (7) weight-update machinery: coincidence pulse programming
(local outer products), with device nonlinearity/endurance the classic pain point; (8) an FPGA
phase sequencer (settle→hold→nudge±→snapshot→update).

**Six-month prototype plan (borrow physics, don't fab — main track: optical).**
*Primary — desktop optical EP machine (Goodman MVM + electronic loop):* one off-the-shelf LCoS SLM
(~2M pixels, $15–25k) holds, with differential encoding, ~1M signed analog weights — the fully
digitally-validated R10 thick block (C=128, 12C² ≈ 200k weights) occupies **one tenth of one SLM**
(5× headroom). Weights (WQ/K/V/O, FFN) static on the SLM during settling; state z (C=128) cycles
through a 128-channel DAC-driven source array → SLM → cylindrical-lens summation → photodiode
array; nonlinearity, T² attention scores (negligible digitally at T=64), Euler integration, and
the λ/gate control law in loop electronics; one loop pass = one Euler step. **Timing is set by
B·T multiplexing, not settling**: each Euler step = B·T·(~6 matrices) MVM passes; at loop rates
0.1–1 MHz and B=4–8, settle ≈ sub-second and a 14k-step training run ≈ hours; SLM refresh once
per training step (60 Hz ample). **Wᵀ for AEP: program the transposed panels alongside W in the
spare SLM area** (reverse-propagation reciprocity remains the Phase-2 elegance; don't gate the
prototype on bidirectional alignment). Precision framing: master weights live fp32 in the digital
shell, the SLM holds a fresh ~8-bit projection each step — standard QAT regime; the open question
is per-pass multiplicative noise + slow drift (speckle/calibration, the optical ~1% floor), which
is exactly what the spec-sheet arsenal (r=0.2 nudging, lock-in/homodyne — the field's native
measurement, same-device contrast cancellation, λ-controller) was built for, and which is
**pre-validated digitally by an optics-noise-model run** (8-bit weight quantization per step +
1–2% multiplicative force-eval noise + drift) before any purchase. Novelty: photonics has in-situ
BP and BP-free local learning (Science 2023 ×2); EP-on-optics exists only as oscillator theory —
"EP-trained transformer on optics" is unclaimed. Budget $20–50k; optics ~2 months, loop
electronics 1–2 months (rehearsed by the PCB track, same parts/skills), calibration+training 2.
*Secondary (one day of email, no more):* Mythic M1076 as a borrowed settle engine — gated solely
on SDK raw-MVM access + incremental writes; flash endurance marginal beyond few-k-step demos.
(Laydevant et al.'s D-Wave EP precedent: reviewers accept rented physics.)
*Fallback / rehearsal (zero-dependency):* board-level reciprocal block (C=8–16, Hopfield Wm
driven bidirectionally + energy attention), digipot/MDAC weights, OTA+cap integrators, Red Pitaya
AC nudge + lock-in, comparator+RC λ loop — small headline, but lands continuous settling, lock-in
contrast readout, and the homeostatic control law in real electronics, and its loop electronics
ARE the optical track's loop electronics. Six months buys no foundry CMOS — but **university cleanrooms (e.g., UIUC HMNTL) are a different
category, and they fabricate exactly the one BOM item money can't buy: the weight arrays.**
Passive BEOL memristor crossbars (bottom electrode / ALD HfOx-TaOx / top electrode, 3–4 masks,
µm linewidth, contact or maskless litho) are the academic-cleanroom comfort zone; the practical
per-array limit is sneak-path-set (~64×64–128×128 for 1R passive with V/2 biasing), and a handful
of tiled arrays covers a C=32 reciprocal block (C=128 ≈ a dozen-array wiring project). FeFET
(three-terminal, on the same line's ferroelectric pedigree) cures sneak paths for a few extra
masks. The algorithm side has already bought insurance for first-batch device quality: Phase-1
keeps fp32 masters with **program-verify writes (only ~6-bit iterative programmability needed —
no pretty pulse physics)**, and 10–50% device mismatch is absorbed by same-device contrast
cancellation + the λ-controller — validated in digital twin runs (per-step N-bit weight
projection + per-pass multiplicative noise + static mismatch; see `--wq_bits/--fnoise/--wmis`).
Discipline: fab ONLY the arrays; anything on Digi-Key stays COTS board-level (student-process
CMOS periphery would be stone-age). This raises the board-track ceiling from digipot (~10³
weights, C=8) to homemade crossbars (10⁴–10⁵ weights, C=32–64) without leaving campus. Execution
for a no-fab-experience team: the standard academic rentals — (1) recipe-owner collaboration
(their senior student runs their existing process; weeks of routine work; co-authorship; you
never gown up), (2) apprenticeship via facility training + staff engineers (executed by a
recruited student), (3) paid staff-run fabrication. Find the recipe owner before booking tool
time; lead the pitch with the device-twin plot ("your first-batch devices suffice — proven").

**Recommended Phase-1 architecture: analog equilibrium core + digital optimizer shell.** Physics
performs only the expensive part (settling + contrast measurement = the ×300–1000 digital
overhead); contrasts are ADC'd out, Adam/schedules/λ-logic stay digital, weights DAC back. This
sidesteps component (7)'s update-nonlinearity pain and the missing analog Adam, at no loss of the
compute advantage. Phase-2: full in-array updates.

**Sizing correction — causal serialization (the assumption that kills the wafer-scale monster):**
naive sizing assumes the whole sequence's state must be physically resident during settling
(K·T·C integrator nodes — hundreds of millions at 8B scale, 1e4–5e4 mm²). Causality removes this:
the free phase settles **token-by-token** (token t's equilibrium depends only on tokens ≤ t; past
states live in an ordinary digital KV-cache), and the nudged phase is the adjoint of a
time-lower-triangular system = an exact **reverse sweep** (upper-triangular back-substitution,
exact to the same order as the AEP linearization). Physical state requirement drops from K·T·C to
**K·C per token-slice** (÷T ≈ ÷2048): ~15 mm² of integrators; sequence caches are ~1 GB of
commodity DRAM. Readout also streams per token-slice (~10⁵ values/token, batch-accumulated in
charge domain) — ADC throughput lands at the standard IMC design point, wall-clock days for an
8B-Chinchilla run at MWh-scale energy. Remaining big item: weight arrays only — 0.6B ≈ 2–4 chips
@28nm (university-consortium scale, $5–20M staged program); 8B ≈ 5–8 reticle dies @7nm (gen-3).
New throughput consideration: serialized operation makes τ_settle the rate limit, and the spring
chain's ~K² mixing tax favors **shallow-wide stacks (K=4–8)** — which the ladder data already
supports (thick single block ≈ same-shape BP). Program staging: MPW single-block demo (33M-class,
$1–5M) → 0.6B 2–4-chip machine → 8B gen-3. Economics read: capex-dominated, opex→0 — the pitch is
2–3 orders of magnitude energy and edge/continual learning, not cloud-GPU rent replacement.

**Compute reality (digital simulation)**: EP's per-step force-eval budget E ≈ 700–3000 makes full
EP training cost ≈ E/3 ≈ **230–1000× the BP cost** at equal tokens (Chinchilla 20×: 0.6B ⇒ 12B
tokens ⇒ ~4.3e19 BP-FLOP ⇒ ~1e22-class EP-FLOP: H100-cluster scale for one run, years on 4×A6000).
This multiplier is exactly what physical settling eliminates — the algorithm is expensive in
digital silicon and native in physics. Note TinyStories (~0.6B tokens) Chinchilla-matches ~30M
params — precisely the planned "readable stories" demo scale (S4).

## 10. Code map (all on timan1)

- `/tmp/lt_ep/lt_ep_train.py` — main trainer: EQBlock (all four force variants), `ep_step`
  (VF-EP + AEP + optional holomorphic nudge), `bptt_step`, λ controller, caps.
  Key flags: `--mode ep|bptt --attn_mode thick|real|energy|mono --jacreg --jr_floor --res_target
  --res_ema --jr_lrcouple --holo N --hr r --c --T1 --T2 --eps --beta`.
- `/tmp/lt_ep/holo_ep.py` — holomorphic force/softmax/LN/GELU, `holo_a` (Cauchy readout), probe.
- `/tmp/lt_ep/grad_quality.py` — estimator-vs-exact cosine probe (validity threshold measurement).
- `/tmp/lt_ep/solver_wall.py` — plain vs Anderson free-phase convergence per damping level.
- `/tmp/lt_ep/bp_charlm.py` — param-matched standard BP transformer baseline.
- `/home/yurenh2/ept/cet_mvp.py`, `cet_aep.py`, `aep_*.py` — CET reproduction + AEP validation
  (vision side; gradient-fidelity numbers in Sec 3/FINDINGS).
- Run logs: `/tmp/lt_ep/thickep_*.log`, `H2_*.json`.
- Data: `/tmp/lt_ep/data/shakespeare_char/{train,val}.bin, meta.pkl`.

Hardware: 1× RTX A6000 per run (shared node); plain-EP ~2.4 it/s, holo-EP(N=2) ~1.5–2 it/s at
B=32, T=64, C=128, T1=150, T2=20. A 14k-step run ≈ 1.6–2.5 h.