diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-08-06 12:15:30 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-08-06 12:15:30 -0500 |
| commit | aa1f7a38e2a9081cad82ab095171f219209809c6 (patch) | |
| tree | 60194433063d80c1ac0ca91aa75018d1f9d1395e /paper/ONE_PAGE.html | |
| parent | d91cfe4d806f4c1e09c6cb75829a8625ff6506ec (diff) | |
docs: add one-page SDIL summary
Diffstat (limited to 'paper/ONE_PAGE.html')
| -rw-r--r-- | paper/ONE_PAGE.html | 95 |
1 files changed, 95 insertions, 0 deletions
diff --git a/paper/ONE_PAGE.html b/paper/ONE_PAGE.html new file mode 100644 index 0000000..777e38f --- /dev/null +++ b/paper/ONE_PAGE.html @@ -0,0 +1,95 @@ +<!doctype html> +<html lang="en"> +<head> +<meta charset="utf-8"> +<title>SDIL in One Page</title> +<style> + @page { size: Letter; margin: 0.48in 0.58in; } + * { box-sizing: border-box; } + body { margin: 0; color: #17232d; font: 9.4pt/1.24 Arial, sans-serif; } + h1 { margin: 0; color: #17324d; font-size: 23pt; line-height: 1.02; } + .subtitle { color: #176b9a; font-size: 13pt; margin: 2px 0 8px; } + .date { float: right; color: #5c6973; font-size: 8.5pt; margin-top: 4px; } + .idea { background: #eef5f8; border-left: 4px solid #176b9a; + padding: 7px 9px; margin: 0 0 7px; } + h2 { color: #17324d; font-size: 12.5pt; margin: 7px 0 2px; } + p { margin: 2px 0 4px; } + .eq { text-align: center; font: 12.5pt Georgia, serif; margin: 4px 0; } + table { width: 100%; border-collapse: collapse; margin: 5px 0; font-size: 8.7pt; } + th, td { padding: 3px 5px; border-bottom: 1px solid #d2dbe0; text-align: right; } + th:first-child, td:first-child { text-align: left; } + thead th { border-top: 1.5px solid #17324d; border-bottom: 1px solid #17324d; } + tbody tr:last-child td { border-bottom: 1.5px solid #17324d; } + ul { margin: 2px 0 0; padding-left: 17px; } + li { margin: 1px 0; } + .two { display: grid; grid-template-columns: 1fr 1fr; gap: 15px; } + footer { margin-top: 7px; padding-top: 4px; border-top: 1px solid #cad3d8; + color: #59666f; font-size: 7.8pt; } +</style> +</head> +<body> + <div class="date">Working summary · 6 August 2026</div> + <h1>Learning from the Unexpected</h1> + <div class="subtitle">Somato-Dendritic Innovation Learning (SDIL)</div> + + <div class="idea"><b>The idea.</b> A neuron's teaching signal contains both useful feedback and + ordinary activity-related bias. Predict the ordinary part from that neuron's own activity, + subtract it, and learn only from what remains.</div> + + <h2>The whole method</h2> + <p>For layer <i>l</i>, fit a local predictor during neutral observations and form the innovation:</p> + <div class="eq">â<sub>l</sub> = P<sub>l</sub>(h<sub>l</sub>), + r<sub>l</sub> = a<sub>l</sub> − â<sub>l</sub>.</div> + <p>Then update each forward weight with information available at that neuron:</p> + <div class="eq">ΔW<sub>l</sub> = η [r<sub>l</sub> ⊙ φ′(u<sub>l</sub>)] + h<sub>l−1</sub><sup>T</sup>.</div> + <p>There is no reverse-mode differentiation, backward computation graph, or transport of + transposed forward weights. The feedback direction can come from causal perturbations or a + reciprocal local-feedback network. The new operation is the per-neuron subtraction, not the + feedback backbone.</p> + + <h2>Current flagship result: standard CIFAR-10 ResNets</h2> + <p>Five paired seeds were run at every depth. SDIL improves as the network gets deeper and remains + close to exact backpropagation (BP).</p> + <table> + <thead><tr><th>Method</th><th>ResNet-20</th><th>ResNet-32</th><th>ResNet-56</th><th>ResNet-56 cost / BP</th></tr></thead> + <tbody> + <tr><td><b>SDIL</b></td><td>91.584</td><td>92.254</td><td><b>92.760</b></td><td>1.331×</td></tr> + <tr><td>BP</td><td>—</td><td>—</td><td>92.632</td><td>1.000×</td></tr> + <tr><td>Clean reciprocal feedback</td><td>—</td><td>—</td><td>92.670</td><td>—</td></tr> + <tr><td>DFA</td><td>—</td><td>—</td><td>30.850</td><td>—</td></tr> + </tbody> + </table> + <p>Numbers are mean test accuracy in percent. Every SDIL seed improves from ResNet-20 to + ResNet-56; the mean gain is 1.176 points. Its early-layer teaching direction has 0.999423 cosine + agreement with the exact descent direction.</p> + + <div class="two"> + <div> + <h2>The result that isolates the subtraction</h2> + <p>With strong activity-predictable interference, raw feedback and a magnitude-matched raw + control reach 10.38% and 10.31% accuracy. SDIL reaches <b>97.35%</b>. Across five seeds, its + paired gain over the magnitude-matched control is <b>87.038 ± 0.607 points</b>. Matching signal + size does not explain the result; subtraction changes the direction used for learning.</p> + </div> + <div> + <h2>Accuracy versus cost</h2> + <p>In the matched author-code VGG16 comparison, SDIL reaches <b>90.70%</b> validation accuracy + in <b>1.24 h</b>. Dual Prop reaches 92.38% in 5.97 h. Clean reciprocal feedback reaches 90.86% + in 1.08 h. SDIL is much cheaper than Dual Prop at similar accuracy, but it does not beat every + method on clean data.</p> + </div> + </div> + + <h2>What would make the paper stronger</h2> + <ul> + <li>Show the same failure and recovery inside strong contrastive learners; the frozen Dual Prop bias screen is running now.</li> + <li>Finish the matched CNN, ResNet, and Transformer comparison; report accuracy, runtime, memory, and update cost together.</li> + <li>Test bias that drifts or is only partly predictable. Random noise is a negative control because this subtraction cannot remove it.</li> + </ul> + + <footer>Inspired by neuron-specific somato-dendritic residuals reported by Francioni, Zhang, + Ahrens and Harnett, <i>Nature</i> (2026). All numbers above come from frozen, retained runs in this + repository; failed branches are not removed.</footer> +</body> +</html> |