path: root/README.md

# Somato-Dendritic Innovation Learning (SDIL)

SDIL is a non-backpropagation learning project motivated by the neuron-specific
somato-dendritic residuals reported by Harnett and colleagues. Its central
algorithmic hypothesis is narrow: a mixed apical compartment should teach from
the component that is unexpected given the same neuron's ordinary somatic
state, rather than from raw apical activity.

The archived 81-cell digital crossover packet is documented in
[`COLLABORATOR_ONBOARDING.md`](COLLABORATOR_ONBOARDING.md), including its
method boundary, completed cells, environment, and restart policy.

For hidden population `l`, the implemented rule is

```text
a_l       = A_l c + ordinary apical traffic
a_hat_l   = P_l(h_l), fitted during neutral periods
r_l       = a_l - a_hat_l
Delta W_l = eta (r_l * local postsynaptic gain) h_(l-1)^T.
```

## Current paper direction

The current paper studies predictable error in a local teaching channel. Its
three experimental parts are:

1. add the same innovation subtraction to digital Dual Propagation,
   Equilibrium Propagation, standard coupled learning, and overclamped coupled
   learning;
2. scale a digital Contrastive Local Learning Network (CLLN) from 32 to 2,048
   learnable edges under fixed component imperfection, comparing clean
   learning, same-RMS zero-mean noise, raw imperfection, static calibration,
   and SDIL;
3. test the update in the nonlinear CLLN with published component-error scales,
   nonideal local correlated-double-sampling, and a SPICE check of the local
   sample/hold/subtract primitive.

The CLLN task, circuit equations, released task panel, and component-error
scales follow [Dillavou et al. (2025)](https://arxiv.org/abs/2505.22887). The
size ladder extends the same periodic circuit construction while keeping the
released 4-by-4 tasks as its smallest endpoint.

The current one-line method is

```text
r = measured local teaching signal - instruction-off local prediction
Delta w = eta * r * local eligibility.
```

The predictor can be a fitted local function or a same-state neutral sample.
It reads no BP gradient, device-error constant, downstream weight, or global
task loss. Neutral measurements and their cost are counted.

The completed transfer evidence is: Dual Propagation recovers from 9.40% to
82.92% validation accuracy against 82.86% clean; EP recovers from 31.38% to
74.52% against 76.26% clean; and the nonlinear 4-by-4 CLLN recovers from
25.42% mean classification error to 0% on 120 untouched task/device pairs.
The confirmed digital CLLN ladder contains 40 tasks, three device draws, six
sizes, and five methods. From 32 to 2,048 edges, static-calibration error grows
from 6.15% to 27.19%, while SDIL grows from 0% to 2.29% against 1.56% clean at
the largest size. SDIL reduces the excess final-error growth slope by 96.2%
and stable-failure growth by 89.5% relative to static calibration. At 2,048
edges it uses 23.3% fewer local updates to the censored target but 52.8% more
local scalar reads because each update includes a neutral measurement.
Increasing static calibration from 16 to 256 instruction-off observations per
edge does not repair the largest network: final error changes from 27.19% to
30.31%, and the paired task-bootstrap interval favors the smaller budget.
The separate overclamp ladder gives the same conclusion: at 2,048 edges,
clean overclamp, imperfect overclamp, and overclamp plus SDIL have 0.63%,
46.88%, and 0.83% final error. SDIL lowers its excess final-error growth slope
by 99.0% and stable-failure growth by 98.2%.

Start with [`THREE_PART_EVIDENCE.md`](THREE_PART_EVIDENCE.md) for the evidence
map and [`CLLN_SCALING.md`](CLLN_SCALING.md) for the frozen scaling protocol.
The current hardware-realistic figure is
[`figure_physical_hardware_evidence_confirmation.pdf`](results/figs/figure_physical_hardware_evidence_confirmation.pdf).
The confirmed digital scaling figure is
[`figure_clln_scaling_confirmation.pdf`](results/figs/figure_clln_scaling_confirmation.pdf).
The additive overclamp scaling figure is
[`figure_clln_overclamp_scaling_confirmation.pdf`](results/figs/figure_clln_overclamp_scaling_confirmation.pdf).

The standard-ResNet stability branch additionally uses a fast paired neutral
observation to project any *remaining* affine neutral residual off current
soma before plasticity. This local microphase is explicitly counted; it is not
described as single-phase or free.

`A_l` is calibrated from antithetic node perturbations without reverse-mode
differentiation or weight transport. That learned-feedback mechanism is
inherited from Lansdell, Prakash, and Kording (2020); it is not claimed as
novel. The candidate contribution is the neutral-period, per-cell innovation
operation under mixed apical traffic, together with its theory, causal audit,
and scaling behavior. See `NOVELTY.md` for the exact prior-art boundary.

## Previous clean-scaling program (retained audit trail)

The results below remain reproducible and useful as baseline evidence. Clean
ResNet scaling in this program comes from the reciprocal Kolen--Pollack
backbone, so it is not used as evidence that residualization itself causes
scaling in the current paper.

- On flattened CIFAR-10, SDIL loses only `0.214 +/- 0.349` accuracy points from
  hidden depth 5 to 60, while DFA's early-layer teaching alignment falls from
  `0.514` to `0.047`. This is depth preservation on a depth-flat task, not yet
  evidence that added depth is useful.
- With strong soma-predictable apical traffic, raw and norm-matched raw
  learning fall to `10.38%` and `10.31%`, while innovation learning retains
  `97.35%`. The paired innovation gain over norm-matched raw is
  `87.038 +/- 0.607` points across five seeds.
- Direct, unamortized node perturbation passes the controlled useful-depth
  diagnosis, but costs `68.4x` ordinary forward-equivalent work. The learned
  apical vectorizer, rather than the local eligibility rule, is the current
  useful-depth bottleneck.
- A low-query K1/every-4 calibration protocol retains `112.9%` of the K16 gain
  over DFA with 16x fewer logical queries, 11x less calibration work, and 5.3x
  less total forward-equivalent work in the frozen MLP protocol.
- The frozen standard-ResNet funnel passed its BP reference and short screen:
  BP reached `91.62%`, and short-run channel-gated SDIL reached `41.98%`
  versus tuned DFA's `37.16%`. The full A3 run then failed decisively: SDIL
  became nonfinite at epoch 89 and ended at chance, while DFA remained finite
  at `33.06%`. A4 was therefore not opened and no confirmation test seed was
  touched.
- The post-failure representable-subspace estimator passed its exact mechanics
  tests and improved frozen-forward early/all-layer alignment from
  `0.0011/0.0117` to `0.0072/0.0527` at matched query count. It nevertheless
  missed the frozen `0.01` early-layer and improvement gates, so no v2 full
  ResNet run was launched.
- Direct vectorizer-space perturbation then passed its exact mechanics and
  synthetic variance audit, but V3-1 again failed: early alignment was
  `0.00714` versus matched V2's `0.00721`, although all-layer alignment rose to
  `0.06258`. V3 full training therefore remained closed.
- Residual response mirroring passes its short gate but fails its sole full
  run at 10% with NaN validation loss. Its endpoint Q/W cosine nevertheless
  reaches 0.999998, exposing final alignment as an inadequate certificate for
  an intermittent tracker. The separate reciprocal KP short gate reaches
  82.66%, and its frozen full gate reaches 91.26% versus matched BP's 91.62%
  with 0.9997 late feedback cosine. The subsequent frozen mixed-traffic screen
  nevertheless makes raw, norm-matched raw, and innovation all nonfinite in
  epoch 1. A separately frozen operator-stability branch then rules out fixed
  predictor margins and passes a dynamic paired-neutral controller: its D1
  352-step trajectory is finite, and D2 reaches `83.58%` after 20 epochs versus
  clean KP's `82.66%` and the failed mixed-traffic controls' `10%`, at `1.3261x`
  BP MACs and zero task-loss queries. This is still one short validation seed;
  the predeclared 200-epoch D3 run subsequently passes all 19 gates at `91.18%`
  versus BP's `91.62%` and clean KP's `91.26%`, with `0.9994` early alignment,
  zero task-loss queries, and `1.3261x` BP MACs.  This raises the strict score
  to 6/10.  The independently frozen paired five-seed D4 test panel then
  passes every gate: dynamic innovation reaches `91.584%` mean test accuracy
  versus clean KP's `91.388%`, wins the mean pairing by `0.196` points, and
  has a clean-minus-dynamic one-sided 95% upper bound of only `0.131` points.
  Every dynamic seed reaches at least `91.51%`, mean early alignment is
  `0.999687`, and all trajectory, projection, leakage, query, hardware, MAC,
  memory, split, and test-isolation checks pass.  This untouched confirmation
  establishes the strict 7/10 accept bar; D4 alone does not establish that
  added standard-network depth is useful.
- Native author-code fidelity is complete. BurstCCN reaches `80.10%` at its
  validation-selected epoch versus published `82.97 +/- 0.21%`; Dual Prop
  reaches `92.46%` versus published `92.41 +/- 0.07%`. Their audited walls are
  `15451.3 s` and `23119.8 s`, respectively, and they are not presented as
  matched-compute points.

The broad endogenous-traffic gate and the Harnett desired-velocity/online-control
screen failed. Those results are retained and explicitly constrain the paper:
SDIL does not currently explain the complete reported dendritic population
signature or online control, and the residual mechanism is supported for
traffic predictable from the chosen somatic statistic rather than arbitrary
top-down context. `ROADMAP.md` and `ORAL_B.md` contain the frozen gates and
complete negative branches.
A temporal-difference recovery separates perturbation-learned causal role from
within-episode performance velocity. Its development gate selects `eta=0.1`
with 98.05% worst-task success, and its untouched 30-record confirmation
retains 99.53% mean success, positive sign inversion in all 30 records, and a
0.639 velocity-over-error correlation advantage. The complete R2 gate
nevertheless fails: residual outcome decoding is only 47.33%, residuals trail
soma by 4.32 points, and the longitudinal correlation is `-0.013`. Thus the
recovery sharpens the boundary--causal-role temporal-difference plasticity
works in this synthetic task, but the broader Harnett-like population
vectorization signature is not established. At that stage the strict score
remained 7/10 and the old oral-A depth panel stayed sealed.

An independent oral-B-v2 then adds an explicit terminal reward/timeout phase,
a local linear TD critic, and temporal eligibility traces. Its first frozen
grid is retained as a cold-start failure, and a fixed-target recovery is
retained after one new seed remains at ceiling. The final algorithm is frozen
before a label-free psychometric calibration: cursor-max quantiles from 512
outcome-free trials define targets for independent rewarded/timeout trials.
The complete untouched 6-task by 5-model confirmation passes every clustered
gate: 100% task success, 0.976 mean learned-role cosine, 0.063
residual--soma correlation, 54.37% surrounding-state decoding, 50.09%
rewarded trials, 99.83% terminal outcome decoding, a 0.400 acute outcome-lesion
drop in role separation, and 30/30 positive causal signs. This raises the
formal milestone to 8/10 and permits only a separately frozen oral-A-v2
protocol; the old depth panel remains closed.

That separately frozen oral-A-v2 protocol is now complete. Its audited 60
records cross BP, tuned DFA, clean reciprocal KP, and dynamic SDIL over
ResNet-20/32/56 with five paired seeds. Dynamic SDIL rises
`91.584% -> 92.254% -> 92.760%`; all five depth-20-to-56 pairs improve, with a
mean gain of `1.176` points. ResNet-56 SDIL reaches `92.760%` versus `92.632%`
BP, `92.670%` clean KP, and `30.850%` DFA while retaining `0.999423`
early-third alignment at `1.331x` the matched BP MAC estimate. This establishes
the internal 9/10 standard-depth milestone. It does not erase the failed old
branch or show that residualization, rather than the inherited reciprocal
backbone, causes clean-task scaling.

The next matched crossover is mechanically registered as nine distinct
architecture/size points times nine methods: miniCNN/VGGlike/VGG16,
ResNet-20/32/56, and decoder-Transformer-4/8/12, each with BP, ordinary FA,
DFA, PEPITA, Forward--Forward, EP, Dual Propagation, clean KP, and SDIL. All 81
cells are mandatory; adding a width, context, or depth point adds all nine
methods rather than an SDIL-only extension.

The complete Plain-CNN P2 panel now passes its 27/27 audit. SDIL scales from
`82.90%` to `89.58%` to `90.70%` validation accuracy from miniCNN through
VGG16, while ordinary FA falls from `68.14%` to `33.80%` and DFA becomes
nonfinite at VGG16. Dual Propagation reaches `92.38%` on VGG16 but takes
`5.97` hours versus SDIL's `1.24`; clean KP slightly dominates SDIL there at
`90.86%` in `1.08` hours. Thus the first family supports scaling and a
cost advantage over iterative strong baselines, but not global Pareto
dominance. The ResNet and Transformer crossover cells were gated on their
complete validation-only selectors; both gates have now passed and are
tracked. ResNet is complete at 19/19 candidates and Transformer at 27/27,
with all nine selected rates frozen before the full-size endpoints. The
GTX-1080 ResNet R2 panel is active, and the independent A6000 81-cell run
packet is open at tag `a6000-x1-v1`.

## Publication-facing artifacts

- `RESULTS.md`: audited positive and negative results;
- `COLLABORATOR_ONBOARDING.md`: complete A6000 81-cell replication packet;
- `THEORY.md`: estimator variance, descent conditions, conditional innovation,
  timescales, and hardware-independent cost;
- `BASELINES.md`: matched and native-author baseline ledger;
- `ROADMAP.md`: accept/oral evidence gates and their state;
- `MIXED_TRAFFIC.md`: frozen standard-ResNet raw/matched/innovation accept gate;
- `DYNAMIC_INNOVATION.md`: post-failure two-timescale stability theory and
  frozen D1--D4 gates;
- `ORAL_B_RECOVERY.md`: structural diagnosis and mechanics-only temporal-
  difference recovery boundary;
- `ORAL_B_V2.md`, `ORAL_B_V2_RECOVERY.md`, and
  `ORAL_B_V2_CALIBRATED_RECOVERY.md`: retained v2 failures and the passed
  label-free calibrated outcome-surprise protocol;
- `ORAL_A.md`: frozen standard CIFAR ResNet funnel;
- `ORAL_A_V2.md`: frozen post-failure representable-subspace funnel;
- `ORAL_A_V3.md`: frozen vectorizer-space causal-calibration funnel;
- `ORAL_A_RECOVERY_V2.md`: passed calibrated-BCI-gated
  ResNet-20/32/56 scaling protocol;
- `CROSS_ARCHITECTURE_CROSSOVER.md`, `RESNET_CROSSOVER.md`, and
  `TRANSFORMER_CROSSOVER.md`: frozen 81-cell matched-crossover contracts;
- `REVIEW_SCORECARD.md`: adversarial ICLR-style score trajectory;
- `paper/MANUSCRIPT.md`: evidence-bound ICLR working draft;
- `paper/CLAIM_LEDGER.json` and `paper/manuscript_audit.json`: direct bindings
  from manuscript numbers, gate statuses, figures, and claim boundaries to
  their audited source files;
- `results/figs/`: deterministic PDF/PNG main figures, captions, and a source
  hash manifest, including the untouched D4 ResNet-20, oral-B-v2, and
  60-record standard-depth confirmations, plus audited RRM failure and
  dynamic-stability supplements.

The current main figures show the local-method Pareto frontier, credit
assignment versus depth, the load-bearing innovation ablation, and the
untouched standard-ResNet confirmations, including positive ResNet-20-to-56
scaling. The audited complete Plain-CNN crossover is rendered separately in
`results/figs/figure7_plain_cnn_pareto.{pdf,png}`, with local-only and
BP-inclusive frontiers kept distinct. The first three main figures require
exactly seeds 0--4; the ResNet panels require paired test seeds 10--14. Strict
renderers refuse missing, dirty, protocol-mixed, source-drifted, or
gate-inconsistent cells.

## Verification

The lab environment currently used for audited CPU checks is
`/home/yurenh2/miniconda3/envs/ep_pascal`. Run:

```bash
experiments/finalize_accept.sh
```

This regenerates all publication-facing figures and manifests, enforces the
frozen Pareto/scaling, D4, oral-B-v2, and 60-record standard-depth gates,
verifies baseline and protocol mechanics, checks the local-rule smoke tests,
evaluates the theory identities, and rebuilds the audited tables.

The convolutional infrastructure can be checked independently:

```bash
/home/yurenh2/miniconda3/envs/ep_pascal/bin/python3 experiments/cifar_image_smoke.py
/home/yurenh2/miniconda3/envs/ep_pascal/bin/python3 experiments/conv_local_smoke.py
```

These checks cover deterministic CIFAR splitting/augmentation, the standard
CIFAR `6n+2` ResNet topology, exact local convolution and BatchNorm
eligibilities, the BatchNorm-coupled perturbation objective, predictor fitting,
translation-shared feedback, representable-subspace causal calibration, and
parameter/cost accounting.  The latter is a transparent post-A3 development
path: it perturbs the channel-gated basis directly and reproduces the expected
full-field delta rule with far less irrelevant spatial variance.  It does not
retroactively reopen or relabel the failed frozen Oral-A protocol.

The same smoke suite also audits a convolutional hierarchical-FA baseline. Its
random feedback tensors follow the real residual DAG and use local activation
Jacobians, but remain independent of forward weights. An audit-only symmetric
copy reproduces exact hidden gradients to `5.3e-16` relative error and the BP
parameter update to `2.98e-8`; actual HFA never performs that copy.
`HFA_BASELINE.md` freezes its matched ResNet-20 validation screen before any
HFA accuracy endpoint is observed.

The learned-hierarchy development path perturbs the feedback convolution
parameter spaces themselves. One antithetic pair estimates every edge's causal
target moment simultaneously, while the current prediction moment is a local
child/parent correlation. Its BatchNorm-coupled JVP and symmetric-limit delta
rule are executable smoke tests. `ORAL_A_V4.md` freezes the causal-capture and
conditional accuracy gates; no accuracy claim is attached until they pass.

Because the global-scalar V4 gate failed, the next strong inherited comparator
uses normalized local response mirroring. Its observation phase sends Gaussian
parent activity through ordinary local forward synapses; a separate feedback
update sees only the probe/child-response pairs. The smoke suite verifies both
high finite-sample kernel recovery and that changing forward parameters after
observation cannot affect the update. Any scale obtained this way is credited
to the weight-mirror baseline until innovation residualization is shown to be
load-bearing on top of it.
`MIRROR_BASELINE.md` freezes that comparator's CIFAR-prefix capture, short
accuracy, and conditional full-validation gates.

After estimate-then-average WM narrowly failed its short accuracy gate, the
next substantive inherited control uses residual response mirroring. Q predicts
the observed child response and learns only from the local prediction error;
unlike WM, its update is zero for every probe at exact symmetry. The extra
feedback prediction convolution is charged explicitly, and this rule receives
no SDIL novelty credit.
`RESIDUAL_MIRROR.md` freezes its capture and conditional task gates.

The stronger reciprocal Kolen--Pollack substrate is audited separately in
`KP_BASELINE.md`. Its short and full frozen gates pass; the full record reaches
91.26% validation accuracy, 0.9994 early alignment, and 1.326x BP MACs without
feedback loss queries. Before that full endpoint, `MIXED_TRAFFIC.md` froze the
actual Harnett-specific test: raw, norm-matched raw, and innovation under
identical four-times-RMS soma-predictable apical traffic and predictor cost.
Mechanics pass, but the complete MT-1 panel fails: all three signals become
nonfinite in epoch 1 and end at 10.00% validation accuracy. MT-2 and MT-3
remain untouched, closing this standard-ResNet recovery without a weaker
traffic intervention.

The subsequent V3 mechanism estimates the required A/G matrix statistics
directly by perturbing the vectorizer parameter subspace. It remains
forward-only and uses two causal loss queries per direction. Its exact and
BatchNorm JVP identities, delta rule, and matched synthetic variance comparison
are covered by `experiments/conv_local_smoke.py`; no task-level claim is made
until a new frozen gate is completed.

`experiments/finalize_accept.sh` additionally requires strict imports of the
frozen BurstCCN and Dual Propagation author-code runs. Both records now pass;
the accept-bar mechanical audit is green. The original standard-ResNet branch
stopped at its failed A3 validation gate and its A4 panel remains sealed; the
independently authorized v2 recovery is the separate passed 60-record panel.

## Result discipline

Every publication-facing run records the source revision, tracked-tree state,
full arguments, split identity, seed, evaluation count, wall time, query/MAC
accounting, and final finiteness. Development, validation, and untouched
confirmation results are never pooled. Failed gates close their branch instead
of triggering seed deletion or post-hoc threshold changes.

The current formal milestone is `9/10` after the untouched D4, oral-B-v2, and
standard-depth confirmations. A conservative external-review forecast is
`8/10`: the full matched nine-method crossover, original-data biological
validation, and a credit pathway novel beyond inherited perturbation/KP
mechanisms remain open. Scores change only after an audited frozen stage, not
after a pilot or presentation improvement.