1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
|
# Rain EP Dillavou-Imperfection Matrix
## Question
The fixed-offset experiment is only an implementation sanity check. With an
exact neutral observation, subtracting a strictly constant offset is already
an oracle and cannot demonstrate an SDIL advantage. The scientific question
is whether SDIL improves downstream accuracy when local hardware bias changes
with device state and a stored constant is therefore insufficient.
The primary corruption is the model in Dillavou et al., Eq. (7). Every local
parameter update receives a fixed, unknown, parameter-specific offset after
the two-state EP estimate has been formed:
\[
g^{\rm measured}_i = g^{\rm EP}_i + B_i.
\]
`B_i` is fixed across examples, epochs, and beta signs. Its RMS is specified
relative to the first clean local update only to set a reproducible simulation
scale. This normalization is not visible to the learner. The optional
state-drift ratio is zero in all exact-model experiments.
The correction is local and uses no backpropagation. With the teaching input
disabled, the update circuit exposes `B_i`. The predictor stores this neutral
measurement and subtracts it from subsequent updates. Under the strictly
constant model, constant calibration, SDIL innovation, and oracle subtraction
are equivalent after the exact neutral observation. A difference between them
is neither predicted nor claimed, and recovery in this setting is not evidence
for SDIL.
## Frozen author protocol
- author repository: `rain-neuromorphics/energy-based-learning`;
- revision: `6b253fd8a5d267535f58ab79992256ef10031ceb`;
- endpoint: `experiments/rain_ep_bias_train.py`;
- network protocol: `comparative32`;
- FashionMNIST with the author's 32x32 augmentation;
- author ConvHopfieldEnergy32 architecture, gains and per-layer rates;
- beta 0.25, 15 training relaxation iterations, 60 inference iterations;
- batch size 128 and 100-epoch cosine schedule.
## S0 development screen
S0 uses only a fixed training/holdout split with 10,000 training and 2,000
holdout examples. One epoch selects a non-catastrophic offset magnitude from
the frozen grid `0.003, 0.01, 0.03, 0.1`. The same wave includes clean PEP,
random-sign beta, centered EP, and one SDIL arm. S0 is development evidence
and is never reported as a final result. S0 and S0b used batch size 100, not
the author's batch size 128.
S0 did not select a magnitude. Clean PEP reached 57.55% holdout accuracy,
while raw offsets from 0.003 through 0.1 reached 60.0--61.8%. At this short
horizon the fixed offsets were not harmful. Random-sign and centered raw arms
were lower at 50.9% and 53.45%, but their clean controls were absent, so this
does not isolate an offset effect. SDIL reduced the residual update offset to
`1.30e-9` of the clean update. A larger-range S0b screen is required.
S0b freezes the larger grid `0.3, 1, 3, 10`. It selects the smallest finite
ratio that lowers one-epoch positive-EP holdout accuracy by at least ten
points relative to its clean control; if none qualifies, it selects 10. The
second wave then compares random-sign, centered, constant calibration, SDIL,
oracle subtraction, and positive EP with beta `0.5, 1, 2` at that ratio. Clean
random-sign and centered controls run in the first wave. This entire adaptive
screen remains development-only.
S0b selected ratio 1.0. At one epoch, positive EP fell from 57.55% clean to
19.65% raw. Random-sign fell from 48.70% to 8.50%, and centered EP fell from
50.85% to 23.15%. Constant calibration and SDIL were bitwise matched at 59.70%
after one local neutral probe; their residual offset was `6.71e-9` relative to
the clean update. Oracle and clean positive EP were exactly matched at 57.55%.
The stronger-beta proxy was non-monotonic: beta 0.5 reached 54.40%, beta 1.0
reached 8.50%, and beta 2.0 became nonfinite. These are one-seed, one-epoch
development observations, not confirmation results.
## C0 full-data trajectory check
C0 freezes ratio 1.0 before launch and returns to the exact author data and
batch protocol: all 60,000 FashionMNIST training examples, the 10,000-example
test set, batch size 128, and the first 10 epochs of the unchanged 100-epoch
cosine schedule. One seed runs eight cells: positive EP clean/raw/constant/
SDIL, random-sign clean/raw, and centered clean/raw. C0 checks whether the S0b
separation persists beyond a small development subset. It is still not the
multi-seed 100-epoch confirmation.
C0 closed as a branch-selection result. Centered EP was the only stable clean
estimator, rising from 79.20% at epoch 1 to 88.67% at epoch 10. Clean positive
EP fell from 73.15% to chance by epoch 7, and clean random-sign EP fell from
77.52% to chance by epoch 7. Constant calibration and SDIL exactly matched but
inherited the positive-EP collapse. Under the fixed ratio-1 update offset,
positive EP became nonfinite at epoch 2, while random-sign and centered EP
became nonfinite at epoch 1. Therefore positive EP cannot be the confirmation
backbone under this full protocol. The next valid comparison must attach local
correction to the stable centered estimator.
## Confirmation matrix
After S0 freezes one offset magnitude, the full author horizon compares:
| EP estimator | no offset | raw offset | constant calibration | SDIL |
|---|---:|---:|---:|---:|
| positive beta | yes | yes | yes | yes |
| random beta sign | yes | yes | no | no |
| centered EP | yes | yes | no | no |
An oracle subtraction arm checks implementation correctness. A large-beta
EP sweep is reported as a strong-clamp proxy but is not called overclamping:
Dillavou overclamping also changes the output force and update duration, so a
beta sweep alone is not the published method.
The exact constant model establishes the hardware failure and the limit of
beta centering. It cannot establish an advantage over ordinary local offset
calibration. Any claimed SDIL advantage requires a separately labeled
state-dependent extension or real measured device drift.
|
