diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 02:12:22 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 02:12:22 -0500 |
| commit | b9a9524ca559cc30a584779ba39f7764887d7fcf (patch) | |
| tree | 3b74674af1d88d3618da00acb41fdc5da62a6431 /NOVELTY.md | |
| parent | 8df3896601797b043dc06659d729047533b79ca6 (diff) | |
docs: correct SDIL novelty boundary
Diffstat (limited to 'NOVELTY.md')
| -rw-r--r-- | NOVELTY.md | 106 |
1 files changed, 106 insertions, 0 deletions
diff --git a/NOVELTY.md b/NOVELTY.md new file mode 100644 index 0000000..3755595 --- /dev/null +++ b/NOVELTY.md @@ -0,0 +1,106 @@ +# SDIL novelty audit + +This file records the closest-prior-art boundary as of 2026-07-22. It is a claim constraint, not +a literature-survey completeness claim. New evidence may narrow the allowed claims further. + +## Exact overlap with learned node-perturbation feedback + +Lansdell, Prakash, and Kording, [Learning to solve the credit assignment +problem](https://arxiv.org/abs/1906.00889) (ICLR 2020), already train synthetic feedback matrices +from node-perturbation estimates. Their direct-feedback form maps the output error to every hidden +layer and was evaluated in a CIFAR convolutional network. Their public implementation is +[`benlansdell/synthfeedback`](https://github.com/benlansdell/synthfeedback). + +With no ordinary apical traffic and `P=0`, SDIL reduces exactly to this idea (up to sign and tensor +conventions): + +```text +c = output error +r_l = A_l c +q_l = node-perturbation estimate of -dL/dh_l +A_l <- A_l + eta_A (q_l - A_l c) c^T +``` + +Independent noise at multiple hidden layers during a shared noisy forward pass is also present in +the earlier method. SDIL's antithetic Rademacher directions, vectorized direction batching, and +explicit cost accounting are useful estimator/implementation choices, but are not the core +algorithmic novelty. The existing no-traffic depth results therefore establish the strength of a +learned-node-perturbation backbone; by themselves they do not establish a Harnett-specific +contribution. + +The closest exact baseline is consequently the same implementation with `traffic_mode=none`, +`learn_P=0`, and `use_residual=0`. It should be named **learned direct feedback by node +perturbation (Lansdell et al.)**, not SDIL, in comparisons intended to isolate novelty. + +## Overlap with dendritic credit assignment + +Apical/basal segregation, neuron-specific dendritic teaching signals, and local three-factor or +burst-dependent plasticity substantially predate SDIL. In addition, Greedy et al., +[Cell-type-specific cortical feedback coordinates hierarchical credit +assignment](https://www.biorxiv.org/content/10.64898/2026.06.16.732595v1) (BurstCCN, 2026), +explicitly connects a scalable dendritic credit-assignment model to the Francioni/Harnett data. +BurstCCN already: + +- models deviations from balanced dendritic feedback as neuron-specific error signals; +- reproduces the opposite residual signs of the P+ and P- BCI populations and their disruption by + NDNF activation; +- trains convolutional models on CIFAR-10 and ImageNet with plastic feedback; and +- provides an [author implementation](https://github.com/neuralml/BurstCCN-journal). + +Thus neither "apical dendrites carry vector errors", "the model reproduces Harnett Fig. 5", nor +"a dendritic local rule scales to ImageNet" is a sufficient novelty claim. + +## Candidate SDIL contribution + +The candidate contribution is the combination + +```text +a_l = A_l c + t_l mixed apical traffic +P_l(h_l) ~= E[t_l | h_l, neutral] per-cell neutral predictor +r_l = a_l - P_l(h_l) somato-dendritic innovation +Delta W_l proportional to r_l * eligibility_l +``` + +where the subtractive per-cell innovation, rather than raw apical activity, is the teaching +variable. The feedback vectorizer `A_l` may be learned using Lansdell-style causal perturbations; +that component is inherited, not claimed as new. The new empirical question is whether +residualization is load-bearing when teaching signals share an apical channel with ordinary +somatic/contextual feedback. + +Neutral-period fitting is an algorithmic adaptation rather than a direct statement of the Harnett +paper. It addresses an identifiability problem: task-period regression can subtract the predictable +part of `A_l c` whenever output error correlates with somatic state, while neutral-period regression +identifies the traffic relationship without observing the teaching component. + +## Claims currently allowed + +- The inherited learned-feedback backbone preserves performance with depth better than fixed DFA + in the audited flattened-CIFAR MLP setting. +- Controlled pilots show that per-cell residualization can protect learning from soma-coupled and + endogenous top-down traffic; the latter remains a validation-only, single-seed result until C1 is + completed. +- Simultaneous batched perturbations make the inherited calibration practical in this codebase; + a hardware-independent query/FLOP advantage is not yet established. + +## Claims currently forbidden + +- Node perturbation training of `A_l` is novel. +- Simultaneously perturbing all hidden layers is novel. +- The no-traffic scaling panel demonstrates the necessity of somato-dendritic residuals. +- SDIL is the first dendritic method to scale to deep vision tasks. +- Reproducing the already-published Harnett/BurstCCN qualitative signatures is an oral-level + biological contribution. + +## Evidence that can restore a strong paper + +1. Show across independently generated, endogenous traffic families that residualization beats + raw and norm-matched raw feedback, while the no-traffic methods coincide. +2. Compare directly with Lansdell-style learned feedback and BurstCCN under validation-selected, + disclosed compute/query budgets. +3. Prove the conditional-bias/SNR benefit of neutral residualization and test its predicted failure + when traffic depends on the teaching signal after conditioning on soma. +4. For a biological contribution, make and test a prediction not used by Harnett or BurstCCN. Raw + event-level data and analysis code require author coordination; the paper exposes only plotted + source data and states that full data/code are available on request. +5. For a scaling contribution, target standard architectures and a simplicity/cost regime not + already occupied by BurstCCN, rather than treating ImageNet alone as novelty. |
