1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
|
# Modified Kolen--Pollack baseline protocol
## Attribution and boundary
This is the reciprocal local-plasticity mechanism of Akrout et al., *Deep
Learning without Weight Transport* (NeurIPS 2019), not an SDIL contribution.
Their equations 16--18 give forward and feedback synapses equal adjustments
from reciprocal local activity pairs and equal decay, so initially distinct
weights converge without transmitting either weight. The public author
repository is frozen for reference at revision
`688f47addd2131684da1b7829b20365f585eee66`:
<https://github.com/makrout/Deep-Learning-without-Weight-Transport>
The public Python implementation is a fully connected pedagogical version and
explicitly transposes a precomputed update. The implementation here instead
recomputes each reciprocal convolutional correlation from its own cached
parent activity and child teaching field. It follows the actual ResNet
residual DAG, including local ReLU and BatchNorm Jacobians and parameter-free
shortcut adjoints. The reciprocal update never reads a forward weight or a
forward update tensor.
KP is introduced because the frozen residual-response mirror full run tests a
different, intermittent probe-response mechanism. KP is not a cadence or rate
rescue of that branch. No KP task endpoint was generated before this protocol
and its executable analyzer were committed.
## KP-0: mechanics gate
The convolutional smoke suite requires all of:
- independently recomputed forward and reciprocal local directions agree to
absolute error below `1e-14` under the storage/sign convention;
- changing every W and Q value after local activities are fixed changes the
reciprocal update by exactly zero;
- exact `Q=W`, `R=-W_out^T` remains exact after two momentum-plus-decay steps;
- the public KP task step leaves all parameters graph-free;
- every pre-existing convolutional local-gradient and feedback check remains
green.
The implemented errors are all exactly zero in float64.
## KP-1: frozen 20-epoch useful-scale gate
Run exactly one seed-0 ResNet-20 development record on the frozen 45k/5k
CIFAR-10 split: batch 128, standard augmentation, hidden/output LR 0.1,
epoch-100/150 step drops (therefore constant during this screen), no warmup,
momentum 0.9, weight decay `1e-4`, BatchNorm, feedback scale 1, and a 32-example
alignment audit. There is no LR, decay, initialization, or feedback-scale grid.
The already frozen matched BP trajectory has 81.02% validation accuracy at
epoch 20 and `1.09487808e14` linearly scaled training MACs. KP-1 passes only if:
- the record, every epoch loss, and every tracking diagnostic are finite;
- validation accuracy is at least 70% and within 15 points of BP epoch 20;
- final early-third teaching alignment is at least 0.50;
- final mean feedback/forward cosine is at least 0.80 and its epoch-11--20
mean is at least 0.70;
- feedback learning uses zero task-loss queries;
- total MACs, including a separate correlation at every reciprocal synapse,
are at most `1.40x` matched BP.
Failure closes KP without an extra learning-rate/decay screen. Passing opens
one full development run.
## KP-2: conditional full baseline
Copy KP-1 exactly for 200 epochs. KP-2 passes only if it is finite, reaches
88% validation accuracy, retains early alignment 0.80, reaches final mean
feedback/forward cosine 0.95, keeps the epoch-151--200 mean cosine at least
0.95, uses zero task-loss queries, and costs at most `1.40x` the frozen full BP
MACs. It never authorizes test access.
KP remains an inherited baseline even if both gates pass, so it cannot raise
the reviewer score. A KP-2 pass only opens a separately frozen
raw-versus-norm-matched-raw-versus-innovation mixed-traffic experiment. The
somato-dendritic innovation must be load-bearing there to affect the SDIL
paper assessment.
## Audited KP-1 outcome
The sole frozen record from source revision `8f420f2` passes every gate:
| metric | KP-1 | frozen requirement/reference |
|:--|--:|--:|
| epoch-20 validation accuracy | **82.66%** | at least 70%; BP epoch 20 is 81.02% |
| early-third teaching alignment | 0.885583 | at least 0.50 |
| final feedback/forward cosine | 0.901034 | at least 0.80 |
| epoch-11--20 mean feedback cosine | 0.854594 | at least 0.70 |
| MAC ratio to matched BP | 1.3261x | at most 1.40x |
| task-loss queries for feedback | 0 | exactly 0 |
All epoch losses and tracking diagnostics are finite. The run uses
`1.451874816e14` MACs, 2.163 GB peak allocated memory, and 558.1 s on a GTX
1080. Its final mean parameter relative error is still 0.445, so KP does not
require exact weight equality to obtain a strong teaching direction. Unlike
the failed intermittent-mirror trajectory, these alignment gates cover the
constant-LR training interval rather than only a decayed-LR endpoint. KP-2 is
opened without changing any optimizer or feedback setting and subsequently
passes at those exact settings.
## Audited KP-2 outcome
The sole frozen full record from source revision `44b110f` passes all seven
gates:
| metric | KP-2 | frozen requirement/reference |
|:--|--:|--:|
| epoch-200 validation accuracy | **91.26%** | at least 88%; matched BP is 91.62% |
| validation loss | 0.429013 | finite |
| early-third teaching alignment | 0.999397 | at least 0.80 |
| final feedback/forward cosine | 0.999663 | at least 0.95 |
| epoch-151--200 mean feedback cosine | 0.999663 | at least 0.95 |
| final feedback/forward relative error | 0.025458 | diagnostic |
| MAC ratio to matched BP | 1.3261x | at most 1.40x |
| task-loss queries for feedback | 0 | exactly 0 |
Every epoch loss and tracking diagnostic is finite. The run uses
`1.451874816e15` MACs, 2.163 GB peak allocated memory, and 5579.7 s on a GTX
1080. It is 0.36 accuracy points below matched BP while using no reverse-mode
gradient or loss query to train its feedback path. The result opens MT-1 but
does not change the 5/10 reviewer score: reciprocal KP is inherited substrate,
not evidence that somato-dendritic innovation is load-bearing. The subsequently
completed MT-1 panel fails under all three traffic rules, so this opening does
not advance to full validation or test confirmation.
|
