Run the model normally.
+ BACKPROP / GLOBAL +Replay the network backward.
- Tokens pass through distinct, standard Transformer blocks. This is both the free - equilibrium and the deployment-time inference path. -
-EPT-LM is a practical route to train standard Transformer language models without a - global backward pass. During training, a small physical nudge turns local state - differences into weight updates. At inference, the model is an ordinary Transformer. + global backward pass. We gently tug the output toward the correct next token; each + layer learns from its own response. At inference, the model is an ordinary Transformer.
One global activation tape
One local response per layer
- No backward activation tape through the stack. No separate inference architecture. - The same bidirectional operations map naturally onto analog matrix hardware. + Backprop reconstructs a global chain of derivatives in software. EPT-LM asks each + layer a local question: how did you move when the answer was nudged?
- Equilibrium Propagation offers a different interface. The hardware runs twice, - once freely and once with a weak target signal. The difference between those settled - states supplies a local learning signal. The device that performs inference can also - participate directly in training. + Equilibrium Propagation offers a different interface. Run the model, then tug its + answer slightly toward the target. Every layer moves in response, and that local + before-and-after change supplies its learning signal. The device that performs + inference can participate directly in training.
Until now, the evidence stopped at vision-scale demonstrations. EPT-LM brings the
@@ -1256,45 +1293,43 @@
The approach
- The free state is exactly the model’s forward activations. Training adds a weak loss
- signal at the output, lets the layer states re-equilibrate, and reads each block locally.
- Three operations make up the full step.
+ Backpropagation sends one instruction backward through the entire network. EPT-LM
+ perturbs the answer and lets every layer observe its own response. The inference graph,
+ model architecture, and output stay the same.
- Tokens pass through distinct, standard Transformer blocks. This is both the free
- equilibrium and the deployment-time inference path.
-
- A small output nudge propagates through the bidirectional training dynamics. A few
- local relaxation sweeps produce the perturbed state.
+ The forward pass stores the activations of every block. Training then revisits the
+ whole stack in reverse to assemble one coordinated gradient.
- Every weight update comes from the block’s own free-to-nudged difference. The
- training rule does not store or traverse a global backward tape.
+ Pull the next-token output slightly toward the correct answer. Each block measures
+ how its own activity changed and converts that response into a local weight update.
A local update for a standard Transformer.
+ Two ways to train the same model.
+
--
cgit v1.2.3
Run the model normally.
+ BACKPROP / GLOBAL
+ Replay the network backward.
Apply the target once.
-
+ storereverseupdate
+
Update each block locally.
+ EPT-LM / LOCAL
+ Give the answer a small tug.
+ runnudgemeasure locally
+
Hardware fit
- Forward and transpose matrix reads, local state storage, and difference measurements
- are native operations for bidirectional analog arrays.
+ Analog arrays already expose forward reads, transpose reads, and local electrical
+ responses. EPT-LM turns those physical operations into the training interface.