2026-07-15T09:29:43.786622
image/svg+xml
Matplotlib v3.10.8, https://matplotlib.org/
(A) The model — a stock OLMo2-style decoder
Inference is one ordinary forward pass.
logits
l
i
n
e
a
r
r
e
a
d
o
u
t
(
u
n
t
i
e
d
)
W
o
u
t
final RMSNorm
t
r
a
n
s
f
o
r
m
e
r
b
l
o
c
k
(
)
=
×
1
2
f
l
h
z
z
=
+
R
M
S
N
o
r
m
(
A
t
t
n
(
)
)
z
h
h
0
=
+
R
M
S
N
o
r
m
(
S
w
i
G
L
U
(
)
)
causal SDPA · QK-RMSNorm · RoPE
token embedding
42.75M (TinyStories 4k BPE) / 72.11M (FineWeb-Edu 32k BPE)
Nothing in this graph is modified for training: the trained checkpoint
is indistinguishable in form from a conventionally trained model.
(B) Free phase (= inference)
g
i
v
e
e
a
c
h
l
a
y
e
r
a
s
t
a
t
e
(
3
o
f
d
r
a
w
n
)
;
s
e
t
t
l
e
t
h
e
d
i
s
a
g
r
e
e
m
e
n
t
e
n
e
r
g
y
z
L
l
E
z
z
f
z
(
)
=
‖
−
(
)
‖
∑
l
l
l
l
1
2
−
1
2
z
f
z
3
3
2
=
(
)
term = 0
z
f
z
2
2
1
=
(
)
term = 0
z
f
1
1
=
(
e
m
b
)
term = 0
e
m
b
(
)
x
T
h
e
m
i
n
i
m
u
m
i
s
e
x
a
c
t
:
=
0
,
s
t
a
t
e
s
≡
f
o
r
w
a
r
d
a
c
t
i
v
a
t
i
o
n
s
,
E
and one bottom-up pass reaches it.
The free phase adds nothing and changes nothing at inference.
(C) Nudged phase (training only)
a
d
d
t
h
e
l
o
s
s
a
t
s
t
r
e
n
g
t
h
≪
1
a
n
d
s
e
t
t
l
e
a
g
a
i
n
:
β
E
z
β
z
y
(
)
+
⋅
C
E
(
l
o
g
i
t
s
(
)
,
)
3
p
u
l
l
−
∇
C
E
β
(the only place the label enters)
z
z
d
β
3
3
3
=
+
z
z
d
β
2
2
2
=
+
z
z
d
β
1
1
1
=
+
e
m
b
(
)
x
d
J
d
2
3
⊤
3
=
d
J
d
1
2
⊤
2
=
T
h
e
t
o
p
s
t
a
t
e
i
s
p
u
l
l
e
d
t
o
w
a
r
d
l
o
w
e
r
l
o
s
s
;
e
a
c
h
l
a
y
e
r
'
s
m
i
s
m
a
t
c
h
d
l
transmits DOWN through the same weights, and the stack re-settles.
(
=
t
h
e
t
r
a
n
s
p
o
s
e
r
e
a
d
a
b
i
d
i
r
e
c
t
i
o
n
a
l
p
h
y
s
i
c
a
l
d
e
v
i
c
e
p
r
o
v
i
d
e
s
)
J
⊤
(D) The update — a difference measurement between the two settled states
̂
=
∂
(
)
−
∂
(
)
/
(
∂
(
)
≡
0
h
e
r
e
,
s
i
n
c
e
=
0
)
g
E
z
E
z
β
E
z
E
[
]
θ
β
θ
θ
0
0
p
e
r
l
a
y
e
r
:
Δ
∝
⟨
,
∂
(
)
/
∂
⟩
/
θ
d
f
z
θ
β
l
l
l
l
l
−
1
Each layer updates from its own boundary mismatch — no global backward graph, no global tape, no loss
d
e
r
i
v
a
t
i
v
e
s
e
x
c
e
p
t
t
h
e
t
o
p
n
u
d
g
e
.
→
0
g
i
v
e
s
t
h
e
e
x
a
c
t
g
r
a
d
i
e
n
t
;
b
i
a
s
i
s
(
)
;
a
±
t
w
o
-
s
i
d
e
d
r
e
a
d
c
a
n
c
e
l
s
i
t
t
o
(
)
.
β
O
β
β
O
β
2
(E) This is not predictive coding — same energy family, different measurement
· PC (as typically run): ONE settled phase with the target clamped hard; the update uses the raw errors of that single state → finite-clamp
bias. Its "exact-BP" results require freezing predictions during error transport (fixed-prediction) — backprop re-expressed in local variables.
· EP (here): TWO phases and an infinitesimal nudge; the update is a difference quotient in β → bias is measured and controllable, and the
settle stays fully self-consistent — which is what physical hardware actually does.
· The free phase is the zero-reference of the measurement: on analog hardware the subtraction cancels state-independent device offsets.