summaryrefslogtreecommitdiff
path: root/NOVELTY.md
diff options
context:
space:
mode:
authorYurenHao0426 <Blackhao0426@gmail.com>2026-07-22 02:12:22 -0500
committerYurenHao0426 <Blackhao0426@gmail.com>2026-07-22 02:12:22 -0500
commitb9a9524ca559cc30a584779ba39f7764887d7fcf (patch)
tree3b74674af1d88d3618da00acb41fdc5da62a6431 /NOVELTY.md
parent8df3896601797b043dc06659d729047533b79ca6 (diff)
docs: correct SDIL novelty boundary
Diffstat (limited to 'NOVELTY.md')
-rw-r--r--NOVELTY.md106
1 files changed, 106 insertions, 0 deletions
diff --git a/NOVELTY.md b/NOVELTY.md
new file mode 100644
index 0000000..3755595
--- /dev/null
+++ b/NOVELTY.md
@@ -0,0 +1,106 @@
+# SDIL novelty audit
+
+This file records the closest-prior-art boundary as of 2026-07-22. It is a claim constraint, not
+a literature-survey completeness claim. New evidence may narrow the allowed claims further.
+
+## Exact overlap with learned node-perturbation feedback
+
+Lansdell, Prakash, and Kording, [Learning to solve the credit assignment
+problem](https://arxiv.org/abs/1906.00889) (ICLR 2020), already train synthetic feedback matrices
+from node-perturbation estimates. Their direct-feedback form maps the output error to every hidden
+layer and was evaluated in a CIFAR convolutional network. Their public implementation is
+[`benlansdell/synthfeedback`](https://github.com/benlansdell/synthfeedback).
+
+With no ordinary apical traffic and `P=0`, SDIL reduces exactly to this idea (up to sign and tensor
+conventions):
+
+```text
+c = output error
+r_l = A_l c
+q_l = node-perturbation estimate of -dL/dh_l
+A_l <- A_l + eta_A (q_l - A_l c) c^T
+```
+
+Independent noise at multiple hidden layers during a shared noisy forward pass is also present in
+the earlier method. SDIL's antithetic Rademacher directions, vectorized direction batching, and
+explicit cost accounting are useful estimator/implementation choices, but are not the core
+algorithmic novelty. The existing no-traffic depth results therefore establish the strength of a
+learned-node-perturbation backbone; by themselves they do not establish a Harnett-specific
+contribution.
+
+The closest exact baseline is consequently the same implementation with `traffic_mode=none`,
+`learn_P=0`, and `use_residual=0`. It should be named **learned direct feedback by node
+perturbation (Lansdell et al.)**, not SDIL, in comparisons intended to isolate novelty.
+
+## Overlap with dendritic credit assignment
+
+Apical/basal segregation, neuron-specific dendritic teaching signals, and local three-factor or
+burst-dependent plasticity substantially predate SDIL. In addition, Greedy et al.,
+[Cell-type-specific cortical feedback coordinates hierarchical credit
+assignment](https://www.biorxiv.org/content/10.64898/2026.06.16.732595v1) (BurstCCN, 2026),
+explicitly connects a scalable dendritic credit-assignment model to the Francioni/Harnett data.
+BurstCCN already:
+
+- models deviations from balanced dendritic feedback as neuron-specific error signals;
+- reproduces the opposite residual signs of the P+ and P- BCI populations and their disruption by
+ NDNF activation;
+- trains convolutional models on CIFAR-10 and ImageNet with plastic feedback; and
+- provides an [author implementation](https://github.com/neuralml/BurstCCN-journal).
+
+Thus neither "apical dendrites carry vector errors", "the model reproduces Harnett Fig. 5", nor
+"a dendritic local rule scales to ImageNet" is a sufficient novelty claim.
+
+## Candidate SDIL contribution
+
+The candidate contribution is the combination
+
+```text
+a_l = A_l c + t_l mixed apical traffic
+P_l(h_l) ~= E[t_l | h_l, neutral] per-cell neutral predictor
+r_l = a_l - P_l(h_l) somato-dendritic innovation
+Delta W_l proportional to r_l * eligibility_l
+```
+
+where the subtractive per-cell innovation, rather than raw apical activity, is the teaching
+variable. The feedback vectorizer `A_l` may be learned using Lansdell-style causal perturbations;
+that component is inherited, not claimed as new. The new empirical question is whether
+residualization is load-bearing when teaching signals share an apical channel with ordinary
+somatic/contextual feedback.
+
+Neutral-period fitting is an algorithmic adaptation rather than a direct statement of the Harnett
+paper. It addresses an identifiability problem: task-period regression can subtract the predictable
+part of `A_l c` whenever output error correlates with somatic state, while neutral-period regression
+identifies the traffic relationship without observing the teaching component.
+
+## Claims currently allowed
+
+- The inherited learned-feedback backbone preserves performance with depth better than fixed DFA
+ in the audited flattened-CIFAR MLP setting.
+- Controlled pilots show that per-cell residualization can protect learning from soma-coupled and
+ endogenous top-down traffic; the latter remains a validation-only, single-seed result until C1 is
+ completed.
+- Simultaneous batched perturbations make the inherited calibration practical in this codebase;
+ a hardware-independent query/FLOP advantage is not yet established.
+
+## Claims currently forbidden
+
+- Node perturbation training of `A_l` is novel.
+- Simultaneously perturbing all hidden layers is novel.
+- The no-traffic scaling panel demonstrates the necessity of somato-dendritic residuals.
+- SDIL is the first dendritic method to scale to deep vision tasks.
+- Reproducing the already-published Harnett/BurstCCN qualitative signatures is an oral-level
+ biological contribution.
+
+## Evidence that can restore a strong paper
+
+1. Show across independently generated, endogenous traffic families that residualization beats
+ raw and norm-matched raw feedback, while the no-traffic methods coincide.
+2. Compare directly with Lansdell-style learned feedback and BurstCCN under validation-selected,
+ disclosed compute/query budgets.
+3. Prove the conditional-bias/SNR benefit of neutral residualization and test its predicted failure
+ when traffic depends on the teaching signal after conditioning on soma.
+4. For a biological contribution, make and test a prediction not used by Harnett or BurstCCN. Raw
+ event-level data and analysis code require author coordination; the paper exposes only plotted
+ source data and states that full data/code are available on request.
+5. For a scaling contribution, target standard architectures and a simplicity/cost regime not
+ already occupied by BurstCCN, rather than treating ImageNet alone as novelty.