From aa1f7a38e2a9081cad82ab095171f219209809c6 Mon Sep 17 00:00:00 2001 From: YurenHao0426 Date: Thu, 6 Aug 2026 12:15:30 -0500 Subject: docs: add one-page SDIL summary --- paper/ONE_PAGE.html | 95 ++++++++++++++++++++++++++++++++++++++++++++++++++++ paper/ONE_PAGE.pdf | Bin 0 -> 70587 bytes 2 files changed, 95 insertions(+) create mode 100644 paper/ONE_PAGE.html create mode 100644 paper/ONE_PAGE.pdf diff --git a/paper/ONE_PAGE.html b/paper/ONE_PAGE.html new file mode 100644 index 0000000..777e38f --- /dev/null +++ b/paper/ONE_PAGE.html @@ -0,0 +1,95 @@ + + + + +SDIL in One Page + + + +

Working summary · 6 August 2026

Learning from the Unexpected

Somato-Dendritic Innovation Learning (SDIL)

+ +

The idea. A neuron's teaching signal contains both useful feedback and + ordinary activity-related bias. Predict the ordinary part from that neuron's own activity, + subtract it, and learn only from what remains.

+ +

The whole method

For layer l, fit a local predictor during neutral observations and form the innovation:

â_l = P_l(h_l), + r_l = a_l − â_l.

Then update each forward weight with information available at that neuron:

ΔW_l = η [r_l ⊙ φ′(u_l)] + h_l−1^T.

There is no reverse-mode differentiation, backward computation graph, or transport of + transposed forward weights. The feedback direction can come from causal perturbations or a + reciprocal local-feedback network. The new operation is the per-neuron subtraction, not the + feedback backbone.

+ +

Current flagship result: standard CIFAR-10 ResNets

Five paired seeds were run at every depth. SDIL improves as the network gets deeper and remains + close to exact backpropagation (BP).

+ + + + + + + + +

Method	ResNet-20	ResNet-32	ResNet-56	ResNet-56 cost / BP
SDIL	91.584	92.254	92.760	1.331×
BP	—	—	92.632	1.000×
Clean reciprocal feedback	—	—	92.670	—
DFA	—	—	30.850	—

Numbers are mean test accuracy in percent. Every SDIL seed improves from ResNet-20 to + ResNet-56; the mean gain is 1.176 points. Its early-layer teaching direction has 0.999423 cosine + agreement with the exact descent direction.

+ +

The result that isolates the subtraction

With strong activity-predictable interference, raw feedback and a magnitude-matched raw + control reach 10.38% and 10.31% accuracy. SDIL reaches 97.35%. Across five seeds, its + paired gain over the magnitude-matched control is 87.038 ± 0.607 points. Matching signal + size does not explain the result; subtraction changes the direction used for learning.

Accuracy versus cost

In the matched author-code VGG16 comparison, SDIL reaches 90.70% validation accuracy + in 1.24 h. Dual Prop reaches 92.38% in 5.97 h. Clean reciprocal feedback reaches 90.86% + in 1.08 h. SDIL is much cheaper than Dual Prop at similar accuracy, but it does not beat every + method on clean data.

+ +

What would make the paper stronger

Show the same failure and recovery inside strong contrastive learners; the frozen Dual Prop bias screen is running now.
Finish the matched CNN, ResNet, and Transformer comparison; report accuracy, runtime, memory, and update cost together.
Test bias that drifts or is only partly predictable. Random noise is a negative control because this subtraction cannot remove it.

+ + + + diff --git a/paper/ONE_PAGE.pdf b/paper/ONE_PAGE.pdf new file mode 100644 index 0000000..4e73b8c Binary files /dev/null and b/paper/ONE_PAGE.pdf differ -- cgit v1.2.3