1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
|
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>SDIL in One Page</title>
<style>
@page { size: Letter; margin: 0.48in 0.58in; }
* { box-sizing: border-box; }
body { margin: 0; color: #17232d; font: 9.4pt/1.24 Arial, sans-serif; }
h1 { margin: 0; color: #17324d; font-size: 23pt; line-height: 1.02; }
.subtitle { color: #176b9a; font-size: 13pt; margin: 2px 0 8px; }
.date { float: right; color: #5c6973; font-size: 8.5pt; margin-top: 4px; }
.idea { background: #eef5f8; border-left: 4px solid #176b9a;
padding: 7px 9px; margin: 0 0 7px; }
h2 { color: #17324d; font-size: 12.5pt; margin: 7px 0 2px; }
p { margin: 2px 0 4px; }
.eq { text-align: center; font: 12.5pt Georgia, serif; margin: 4px 0; }
table { width: 100%; border-collapse: collapse; margin: 5px 0; font-size: 8.7pt; }
th, td { padding: 3px 5px; border-bottom: 1px solid #d2dbe0; text-align: right; }
th:first-child, td:first-child { text-align: left; }
thead th { border-top: 1.5px solid #17324d; border-bottom: 1px solid #17324d; }
tbody tr:last-child td { border-bottom: 1.5px solid #17324d; }
ul { margin: 2px 0 0; padding-left: 17px; }
li { margin: 1px 0; }
.two { display: grid; grid-template-columns: 1fr 1fr; gap: 15px; }
footer { margin-top: 7px; padding-top: 4px; border-top: 1px solid #cad3d8;
color: #59666f; font-size: 7.8pt; }
</style>
</head>
<body>
<div class="date">Working summary · 6 August 2026</div>
<h1>Learning from the Unexpected</h1>
<div class="subtitle">Somato-Dendritic Innovation Learning (SDIL)</div>
<div class="idea"><b>The idea.</b> A neuron's teaching signal contains both useful feedback and
ordinary activity-related bias. Predict the ordinary part from that neuron's own activity,
subtract it, and learn only from what remains.</div>
<h2>The whole method</h2>
<p>For layer <i>l</i>, fit a local predictor during neutral observations and form the innovation:</p>
<div class="eq">â<sub>l</sub> = P<sub>l</sub>(h<sub>l</sub>),
r<sub>l</sub> = a<sub>l</sub> − â<sub>l</sub>.</div>
<p>Then update each forward weight with information available at that neuron:</p>
<div class="eq">ΔW<sub>l</sub> = η [r<sub>l</sub> ⊙ φ′(u<sub>l</sub>)]
h<sub>l−1</sub><sup>T</sup>.</div>
<p>There is no reverse-mode differentiation, backward computation graph, or transport of
transposed forward weights. The feedback direction can come from causal perturbations or a
reciprocal local-feedback network. The new operation is the per-neuron subtraction, not the
feedback backbone.</p>
<h2>Current flagship result: standard CIFAR-10 ResNets</h2>
<p>Five paired seeds were run at every depth. SDIL improves as the network gets deeper and remains
close to exact backpropagation (BP).</p>
<table>
<thead><tr><th>Method</th><th>ResNet-20</th><th>ResNet-32</th><th>ResNet-56</th><th>ResNet-56 cost / BP</th></tr></thead>
<tbody>
<tr><td><b>SDIL</b></td><td>91.584</td><td>92.254</td><td><b>92.760</b></td><td>1.331×</td></tr>
<tr><td>BP</td><td>—</td><td>—</td><td>92.632</td><td>1.000×</td></tr>
<tr><td>Clean reciprocal feedback</td><td>—</td><td>—</td><td>92.670</td><td>—</td></tr>
<tr><td>DFA</td><td>—</td><td>—</td><td>30.850</td><td>—</td></tr>
</tbody>
</table>
<p>Numbers are mean test accuracy in percent. Every SDIL seed improves from ResNet-20 to
ResNet-56; the mean gain is 1.176 points. Its early-layer teaching direction has 0.999423 cosine
agreement with the exact descent direction.</p>
<div class="two">
<div>
<h2>The result that isolates the subtraction</h2>
<p>With strong activity-predictable interference, raw feedback and a magnitude-matched raw
control reach 10.38% and 10.31% accuracy. SDIL reaches <b>97.35%</b>. Across five seeds, its
paired gain over the magnitude-matched control is <b>87.038 ± 0.607 points</b>. Matching signal
size does not explain the result; subtraction changes the direction used for learning.</p>
</div>
<div>
<h2>Accuracy versus cost</h2>
<p>In the matched author-code VGG16 comparison, SDIL reaches <b>90.70%</b> validation accuracy
in <b>1.24 h</b>. Dual Prop reaches 92.38% in 5.97 h. Clean reciprocal feedback reaches 90.86%
in 1.08 h. SDIL is much cheaper than Dual Prop at similar accuracy, but it does not beat every
method on clean data.</p>
</div>
</div>
<h2>What would make the paper stronger</h2>
<ul>
<li>Show the same failure and recovery inside strong contrastive learners; the frozen Dual Prop bias screen is running now.</li>
<li>Finish the matched CNN, ResNet, and Transformer comparison; report accuracy, runtime, memory, and update cost together.</li>
<li>Test bias that drifts or is only partly predictable. Random noise is a negative control because this subtraction cannot remove it.</li>
</ul>
<footer>Inspired by neuron-specific somato-dendritic residuals reported by Francioni, Zhang,
Ahrens and Harnett, <i>Nature</i> (2026). All numbers above come from frozen, retained runs in this
repository; failed branches are not removed.</footer>
</body>
</html>
|
