diff options
Diffstat (limited to 'sdil')
| -rw-r--r-- | sdil/core.py | 24 |
1 files changed, 13 insertions, 11 deletions
diff --git a/sdil/core.py b/sdil/core.py index 37c7165..cece194 100644 --- a/sdil/core.py +++ b/sdil/core.py @@ -4,11 +4,11 @@ SDIL -- Somato-Dendritic Innovation Learning. A non-backprop learning rule inspired by Harnett et al. 2026 (Nature), "Vectorized instructive signals in cortical dendrites". -The load-bearing idea, faithful to the paper, is NOT "apical dendrites carry -error". That is old (Guerguiev/Sacramento/Payeur/burst credit assignment). The -new algorithmic object is the *somato-dendritic residual*: the teaching signal -is the apical feedback MINUS the part of it that is predictable from the cell's -own somatic activity. Only the innovation drives plasticity. +The Harnett-specific object studied here is the *somato-dendritic residual*: +the teaching signal is the apical feedback MINUS the part of it that is +predictable from the cell's own somatic activity. Only the innovation drives +plasticity. Apical credit signals and learned feedback from node perturbations +both have substantial prior art; see NOVELTY.md for the exact claim boundary. For hidden layer l: u_l = W_l h_{l-1}, h_l = phi(u_l) (somatic / basal forward) @@ -21,10 +21,11 @@ Three learning rules, all local (no weight transport, no reverse-mode graph): dP_l = eta_P * (a_l - P_l h_l) h_l^T (slow; only on neutral periods) eta_P << eta dA_l = eta_A * (q_l - r_l) c_l^T (on perturbation trials; q_l = causal node-pert estimate) -q_l is an amortized node-perturbation estimate of the causal descent direction +q_l is a node-perturbation estimate of the causal descent direction -grad_{h_l} L, obtained by occasionally perturbing h_l and watching the loss. -Learning A_l this way (rather than fixing it random) is what keeps SDIL from -degenerating into DFA. +Learning A_l this way follows the learned synthetic-feedback approach of +Lansdell, Prakash & Kording (ICLR 2020). Relative to fixed DFA it supplies an +adaptive feedback backbone; it is not itself claimed as an SDIL novelty. This module deliberately does NOT use autograd for learning. Autograd is used ONLY inside probes.py to MEASURE the true gradient for alignment diagnostics. @@ -92,9 +93,10 @@ class SDILNet: biologically unproblematic; only hidden layers use the SDIL innovation. feedback signal c: by default the output error e = softmax(logits) - onehot, - broadcast to every hidden layer (dim = n_classes). With A_l learned this is - "amortized node perturbation through apical dendrites"; with A_l fixed random - it reduces to DFA (used as an ablation). + broadcast to every hidden layer (dim = n_classes). With P=0 and no traffic, + learning A_l from node perturbations is the direct learned-feedback method + of Lansdell et al.; with A_l fixed random it reduces to DFA. SDIL's candidate + addition is the per-cell innovation under mixed apical traffic. """ def __init__(self, sizes, act="tanh", device="cpu", seed=0, |