1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
|
# Claim verification: "largest neural network trained without backpropagation"
**Verified 2026-07-11** (web sweep + primary-source close reading). Status: **the claim holds today
at 42.75M with one qualifier; holds unqualified at 1B+; is uncontested-by-an-order-of-magnitude at 3-7B.**
## The claim (Scellier-proof wording)
> "the largest neural network ever trained **end-to-end without backpropagation** — every parameter
> learned by a local two-phase rule, at quality parity with a tuned backprop twin, on a standard
> transformer whose inference is a plain forward pass, with every **trained** component
> analog-compatible."
Wording hygiene: (a) "end-to-end / every parameter" is the load-bearing qualifier (see optical DFA
below); (b) say "every *trained* component analog-compatible" — optimizer state is digital-side by
standing doctrine, same as every analog-training proposal; (c) run record attempts with
`--gate_every 0` (fully BP-free mode, formally audited by `ep_run/test_bp_free.py`).
## Competitor autopsy (primary sources)
| Claimant | What actually happened | Why it does not hold the title |
|---|---|---|
| **Optical DFA** — LightOn-line, arXiv:2409.12965 ("Optical training of large-scale Transformers") | 1.07B GPT-2-shape LM | **Partial coverage**: only ~400M of 1.07B params received ODFA signals; **the last layer is trained by BP** (DFA structurally broadcasts a BP-computed top error through fixed random matrices — a global error broadcast, not a local rule); corpus = Cornell Movie-Dialogs, **1.2M tokens, vocab 1016, context 24**; authors: quality "remains behind the state of the art", "lack[s] logical connections" |
| **EGGROLL / ES** — arXiv:2511.16652 (Oxford/FLAIR/Mila/**NVIDIA**) | "billion-parameter" throughput + RWKV7 experiments | Billion-scale = **fine-tuning of BP-pretrained RWKV7** (post-training). The from-scratch component is **EGG: a D256-L6 integer GRU (~1-2M params), char-level MiniPile, 3.40 bits/byte** — not a transformer, weak quality |
| ES fine-tuning — arXiv:2509.24372 (Cognizant) | full-param ES fine-tuning of billion LLMs | Fine-tuning only; no pretraining |
| DFA original — arXiv:2006.12878 (Launay 2020) | transformers among tasks | Tens-of-M scale; authors report a DFA-vs-BP gap specifically on transformers |
| Forward-Forward / PC / Mono-Forward / CaFo line | MNIST/CIFAR-class | <10M, no LM |
| Physical NN — Science 2023 (adi8474, PhyLL) | physical substrates | Small scale |
| EP line record (Laborieux et al. 2021) | ImageNet-32 ConvNets | ~10-20M; **our 42.75M full-epoch already exceeds the EP-line record** |
## What we hold at each scale
1. **Today (42.75M, done 2026-07-10/11):** largest **fully** backprop-free (all parameters, no BP
anywhere in the training loop incl. readout) language model; only one at quality parity with a
tuned BP twin (4k matched-tuning parity; epoch-scale gap 0.04 with Muon+floor-ramp, stage1b);
only one generating coherent text; standard OLMo2-form transformer, plain-forward inference.
The only larger *number* in the room (optical DFA 1.07B) is partial + BP-readout + toy corpus.
2. **At 1B+:** largest without qualifiers — beats the partial claim on its own terms.
3. **At 3-7B:** an order of magnitude past every claim including partial ones. "EVER" grade.
## Differentiation axes (for related-work / rebuttals)
all-parameters coverage · zero BP in the loop (formally audited mode) · standard architecture ·
standard inference (plain forward) · quality parity vs tuned BP twin · coherent generation ·
gradient exactness (cos ~= 1 vs autograd reference) · analog-compatible trained components
(OLMo2 block analog audit, docs/hardware/HW_RESEARCH_FINDINGS.md).
## Sources
- https://arxiv.org/abs/2409.12965 (+ html v1 experimental section, read 2026-07-11)
- https://arxiv.org/abs/2511.16652 + https://eshyperscale.github.io/
- https://arxiv.org/abs/2509.24372
- https://arxiv.org/abs/2006.12878
- https://www.science.org/doi/10.1126/science.adi8474
- https://github.com/UbiquitousLearning/Backpropagation_Free_Training_Survey
- https://arxiv.org/abs/2511.01061 (forward-only rigorous evaluation, 2025)
## UPDATE 2026-07-11: Kerjan-Hoier-Scellier (Rain AI), arXiv:2606.03584 — the closest relative, and a re-scoping.
"Training a PCN on ImageNet using EP" (June 2026): VGG10 PCN, full ImageNet, centered EP,
top-1 33.81 vs BP 32.35 (top-5 13.23 vs 12.20) — **a real 1.0-1.5pp gap remains, at the hands of
EP's inventor**. Their construction IS the layered-energy bridge (their Eq. 10 == our cascade energy;
free phase == forward pass; single-equilibrium simplification == our single-sided legality; their
mod-PGD/asynchronous traversal ~= our fb solver). Vision classification only; hardware explicitly
punted ("remains unclear whether F_PCN could be efficiently realized in hardware"). Parameter count (computed from their Table 6): conv ~9.2M + dense 25088x2048 ~51.4M + 2048x1000 ~2.0M
= **~62.7M > our 42.75M** => **"largest NN today" is THEIRS — do not use.** Also measured: their
EP/BP wall-clock = 12x centered / 6.7x random (VGG10 ImageNet: 18d/10d vs BP 36h, single A100) vs
OUR 3.2x — the solver-efficiency differentiator, measured on both sides.
RE-SCOPED claims that hold today: largest/first TRANSFORMER fully trained without BP; first
AUTOREGRESSIVE LANGUAGE MODEL; the only zero-gap result (matched-tuning indistinguishability; their
best retains 1.0-1.5pp). At 1B+ the unqualified largest-NN claim reactivates regardless of their
param count. STRATEGIC: same lab as CET, funded by Rain AI + ARIA Scaling Compute; their refs
(mu-PC 100+ layers, Goemaere signal-decay, Qi step-size schedules; Momeni et al. Nature 2025 PNN
review) are our depth-scaling reading list and the venue precedent.
|
