2026-07-15T09:44:53.517354
image/svg+xml
Matplotlib v3.10.8, https://matplotlib.org/
Free phase (= inference)
g
i
v
e
e
a
c
h
l
a
y
e
r
a
s
t
a
t
e
(
3
o
f
d
r
a
w
n
)
;
s
e
t
t
l
e
t
h
e
d
i
s
a
g
r
e
e
m
e
n
t
e
n
e
r
g
y
z
L
l
E
z
z
f
z
(
)
=
‖
−
(
)
‖
∑
l
l
l
l
1
2
−
1
2
z
f
z
3
3
2
=
(
)
term = 0
z
f
z
2
2
1
=
(
)
term = 0
z
f
1
1
=
(
e
m
b
)
term = 0
e
m
b
(
)
x
T
h
e
m
i
n
i
m
u
m
i
s
e
x
a
c
t
:
=
0
,
s
t
a
t
e
s
≡
f
o
r
w
a
r
d
a
c
t
i
v
a
t
i
o
n
s
,
E
and one bottom-up pass reaches it.
The free phase adds nothing and changes nothing at inference.
Nudged phase (training only)
a
d
d
t
h
e
l
o
s
s
a
t
s
t
r
e
n
g
t
h
≪
1
a
n
d
s
e
t
t
l
e
a
g
a
i
n
:
β
E
z
β
z
y
(
)
+
⋅
C
E
(
l
o
g
i
t
s
(
)
,
)
3
p
u
l
l
−
∇
C
E
β
(the only place the label enters)
z
z
d
β
3
3
3
=
+
z
z
d
β
2
2
2
=
+
z
z
d
β
1
1
1
=
+
e
m
b
(
)
x
d
J
d
2
3
⊤
3
=
d
J
d
1
2
⊤
2
=
T
h
e
t
o
p
s
t
a
t
e
i
s
p
u
l
l
e
d
t
o
w
a
r
d
l
o
w
e
r
l
o
s
s
;
e
a
c
h
l
a
y
e
r
'
s
m
i
s
m
a
t
c
h
d
l
transmits DOWN through the same weights, and the stack re-settles.
(
=
t
h
e
t
r
a
n
s
p
o
s
e
r
e
a
d
a
b
i
d
i
r
e
c
t
i
o
n
a
l
p
h
y
s
i
c
a
l
d
e
v
i
c
e
p
r
o
v
i
d
e
s
)
J
⊤
The update — a difference measurement between the two settled states
̂
=
∂
(
)
−
∂
(
)
/
(
∂
(
)
≡
0
h
e
r
e
,
s
i
n
c
e
=
0
)
g
E
z
E
z
β
E
z
E
[
]
θ
β
θ
θ
0
0
p
e
r
l
a
y
e
r
:
Δ
∝
⟨
,
∂
(
)
/
∂
⟩
/
θ
d
f
z
θ
β
l
l
l
l
l
−
1
Each layer updates from its own boundary mismatch — no global backward graph, no global tape, no loss derivatives except the top nudge.