path: root/README.md

# Somato-Dendritic Innovation Learning (SDIL)

SDIL is a non-backpropagation learning project motivated by the neuron-specific
somato-dendritic residuals reported by Harnett and colleagues. Its central
algorithmic hypothesis is narrow: a mixed apical compartment should teach from
the component that is unexpected given the same neuron's ordinary somatic
state, rather than from raw apical activity.

External A6000 collaborators running the complete 81-cell matched crossover
should begin with
[`COLLABORATOR_ONBOARDING.md`](COLLABORATOR_ONBOARDING.md). It contains the
method boundary, current positive and negative results, environment/data
setup, frozen matrix, restart policy, and the single entry-point command.

For hidden population `l`, the implemented rule is

```text
a_l       = A_l c + ordinary apical traffic
a_hat_l   = P_l(h_l), fitted during neutral periods
r_l       = a_l - a_hat_l
Delta W_l = eta (r_l * local postsynaptic gain) h_(l-1)^T.
```

The standard-ResNet stability branch additionally uses a fast paired neutral
observation to project any *remaining* affine neutral residual off current
soma before plasticity. This local microphase is explicitly counted; it is not
described as single-phase or free.

`A_l` is calibrated from antithetic node perturbations without reverse-mode
differentiation or weight transport. That learned-feedback mechanism is
inherited from Lansdell, Prakash, and Kording (2020); it is not claimed as
novel. The candidate contribution is the neutral-period, per-cell innovation
operation under mixed apical traffic, together with its theory, causal audit,
and scaling behavior. See `NOVELTY.md` for the exact prior-art boundary.

## Current audited evidence

- On flattened CIFAR-10, SDIL loses only `0.214 +/- 0.349` accuracy points from
  hidden depth 5 to 60, while DFA's early-layer teaching alignment falls from
  `0.514` to `0.047`. This is depth preservation on a depth-flat task, not yet
  evidence that added depth is useful.
- With strong soma-predictable apical traffic, raw and norm-matched raw
  learning fall to `10.38%` and `10.31%`, while innovation learning retains
  `97.35%`. The paired innovation gain over norm-matched raw is
  `87.038 +/- 0.607` points across five seeds.
- Direct, unamortized node perturbation passes the controlled useful-depth
  diagnosis, but costs `68.4x` ordinary forward-equivalent work. The learned
  apical vectorizer, rather than the local eligibility rule, is the current
  useful-depth bottleneck.
- A low-query K1/every-4 calibration protocol retains `112.9%` of the K16 gain
  over DFA with 16x fewer logical queries, 11x less calibration work, and 5.3x
  less total forward-equivalent work in the frozen MLP protocol.
- The frozen standard-ResNet funnel passed its BP reference and short screen:
  BP reached `91.62%`, and short-run channel-gated SDIL reached `41.98%`
  versus tuned DFA's `37.16%`. The full A3 run then failed decisively: SDIL
  became nonfinite at epoch 89 and ended at chance, while DFA remained finite
  at `33.06%`. A4 was therefore not opened and no confirmation test seed was
  touched.
- The post-failure representable-subspace estimator passed its exact mechanics
  tests and improved frozen-forward early/all-layer alignment from
  `0.0011/0.0117` to `0.0072/0.0527` at matched query count. It nevertheless
  missed the frozen `0.01` early-layer and improvement gates, so no v2 full
  ResNet run was launched.
- Direct vectorizer-space perturbation then passed its exact mechanics and
  synthetic variance audit, but V3-1 again failed: early alignment was
  `0.00714` versus matched V2's `0.00721`, although all-layer alignment rose to
  `0.06258`. V3 full training therefore remained closed.
- Residual response mirroring passes its short gate but fails its sole full
  run at 10% with NaN validation loss. Its endpoint Q/W cosine nevertheless
  reaches 0.999998, exposing final alignment as an inadequate certificate for
  an intermittent tracker. The separate reciprocal KP short gate reaches
  82.66%, and its frozen full gate reaches 91.26% versus matched BP's 91.62%
  with 0.9997 late feedback cosine. The subsequent frozen mixed-traffic screen
  nevertheless makes raw, norm-matched raw, and innovation all nonfinite in
  epoch 1. A separately frozen operator-stability branch then rules out fixed
  predictor margins and passes a dynamic paired-neutral controller: its D1
  352-step trajectory is finite, and D2 reaches `83.58%` after 20 epochs versus
  clean KP's `82.66%` and the failed mixed-traffic controls' `10%`, at `1.3261x`
  BP MACs and zero task-loss queries. This is still one short validation seed;
  the predeclared 200-epoch D3 run subsequently passes all 19 gates at `91.18%`
  versus BP's `91.62%` and clean KP's `91.26%`, with `0.9994` early alignment,
  zero task-loss queries, and `1.3261x` BP MACs.  This raises the strict score
  to 6/10.  The independently frozen paired five-seed D4 test panel then
  passes every gate: dynamic innovation reaches `91.584%` mean test accuracy
  versus clean KP's `91.388%`, wins the mean pairing by `0.196` points, and
  has a clean-minus-dynamic one-sided 95% upper bound of only `0.131` points.
  Every dynamic seed reaches at least `91.51%`, mean early alignment is
  `0.999687`, and all trajectory, projection, leakage, query, hardware, MAC,
  memory, split, and test-isolation checks pass.  This untouched confirmation
  establishes the strict 7/10 accept bar; D4 alone does not establish that
  added standard-network depth is useful.
- Native author-code fidelity is complete. BurstCCN reaches `80.10%` at its
  validation-selected epoch versus published `82.97 +/- 0.21%`; Dual Prop
  reaches `92.46%` versus published `92.41 +/- 0.07%`. Their audited walls are
  `15451.3 s` and `23119.8 s`, respectively, and they are not presented as
  matched-compute points.

The broad endogenous-traffic gate and the Harnett desired-velocity/online-control
screen failed. Those results are retained and explicitly constrain the paper:
SDIL does not currently explain the complete reported dendritic population
signature or online control, and the residual mechanism is supported for
traffic predictable from the chosen somatic statistic rather than arbitrary
top-down context. `ROADMAP.md` and `ORAL_B.md` contain the frozen gates and
complete negative branches.
A temporal-difference recovery separates perturbation-learned causal role from
within-episode performance velocity. Its development gate selects `eta=0.1`
with 98.05% worst-task success, and its untouched 30-record confirmation
retains 99.53% mean success, positive sign inversion in all 30 records, and a
0.639 velocity-over-error correlation advantage. The complete R2 gate
nevertheless fails: residual outcome decoding is only 47.33%, residuals trail
soma by 4.32 points, and the longitudinal correlation is `-0.013`. Thus the
recovery sharpens the boundary--causal-role temporal-difference plasticity
works in this synthetic task, but the broader Harnett-like population
vectorization signature is not established. At that stage the strict score
remained 7/10 and the old oral-A depth panel stayed sealed.

An independent oral-B-v2 then adds an explicit terminal reward/timeout phase,
a local linear TD critic, and temporal eligibility traces. Its first frozen
grid is retained as a cold-start failure, and a fixed-target recovery is
retained after one new seed remains at ceiling. The final algorithm is frozen
before a label-free psychometric calibration: cursor-max quantiles from 512
outcome-free trials define targets for independent rewarded/timeout trials.
The complete untouched 6-task by 5-model confirmation passes every clustered
gate: 100% task success, 0.976 mean learned-role cosine, 0.063
residual--soma correlation, 54.37% surrounding-state decoding, 50.09%
rewarded trials, 99.83% terminal outcome decoding, a 0.400 acute outcome-lesion
drop in role separation, and 30/30 positive causal signs. This raises the
formal milestone to 8/10 and permits only a separately frozen oral-A-v2
protocol; the old depth panel remains closed.

That separately frozen oral-A-v2 protocol is now complete. Its audited 60
records cross BP, tuned DFA, clean reciprocal KP, and dynamic SDIL over
ResNet-20/32/56 with five paired seeds. Dynamic SDIL rises
`91.584% -> 92.254% -> 92.760%`; all five depth-20-to-56 pairs improve, with a
mean gain of `1.176` points. ResNet-56 SDIL reaches `92.760%` versus `92.632%`
BP, `92.670%` clean KP, and `30.850%` DFA while retaining `0.999423`
early-third alignment at `1.331x` the matched BP MAC estimate. This establishes
the internal 9/10 standard-depth milestone. It does not erase the failed old
branch or show that residualization, rather than the inherited reciprocal
backbone, causes clean-task scaling.

The next matched crossover is mechanically registered as nine distinct
architecture/size points times nine methods: miniCNN/VGGlike/VGG16,
ResNet-20/32/56, and decoder-Transformer-4/8/12, each with BP, ordinary FA,
DFA, PEPITA, Forward--Forward, EP, Dual Propagation, clean KP, and SDIL. All 81
cells are mandatory; adding a width, context, or depth point adds all nine
methods rather than an SDIL-only extension.

The complete Plain-CNN P2 panel now passes its 27/27 audit. SDIL scales from
`82.90%` to `89.58%` to `90.70%` validation accuracy from miniCNN through
VGG16, while ordinary FA falls from `68.14%` to `33.80%` and DFA becomes
nonfinite at VGG16. Dual Propagation reaches `92.38%` on VGG16 but takes
`5.97` hours versus SDIL's `1.24`; clean KP slightly dominates SDIL there at
`90.86%` in `1.08` hours. Thus the first family supports scaling and a
cost advantage over iterative strong baselines, but not global Pareto
dominance. ResNet and Transformer crossover cells remain gated on their
complete validation-only selectors.

## Publication-facing artifacts

- `RESULTS.md`: audited positive and negative results;
- `COLLABORATOR_ONBOARDING.md`: complete A6000 81-cell replication packet;
- `THEORY.md`: estimator variance, descent conditions, conditional innovation,
  timescales, and hardware-independent cost;
- `BASELINES.md`: matched and native-author baseline ledger;
- `ROADMAP.md`: accept/oral evidence gates and their state;
- `MIXED_TRAFFIC.md`: frozen standard-ResNet raw/matched/innovation accept gate;
- `DYNAMIC_INNOVATION.md`: post-failure two-timescale stability theory and
  frozen D1--D4 gates;
- `ORAL_B_RECOVERY.md`: structural diagnosis and mechanics-only temporal-
  difference recovery boundary;
- `ORAL_B_V2.md`, `ORAL_B_V2_RECOVERY.md`, and
  `ORAL_B_V2_CALIBRATED_RECOVERY.md`: retained v2 failures and the passed
  label-free calibrated outcome-surprise protocol;
- `ORAL_A.md`: frozen standard CIFAR ResNet funnel;
- `ORAL_A_V2.md`: frozen post-failure representable-subspace funnel;
- `ORAL_A_V3.md`: frozen vectorizer-space causal-calibration funnel;
- `ORAL_A_RECOVERY_V2.md`: passed calibrated-BCI-gated
  ResNet-20/32/56 scaling protocol;
- `CROSS_ARCHITECTURE_CROSSOVER.md`, `RESNET_CROSSOVER.md`, and
  `TRANSFORMER_CROSSOVER.md`: frozen 81-cell matched-crossover contracts;
- `REVIEW_SCORECARD.md`: adversarial ICLR-style score trajectory;
- `paper/MANUSCRIPT.md`: evidence-bound ICLR working draft;
- `paper/CLAIM_LEDGER.json` and `paper/manuscript_audit.json`: direct bindings
  from manuscript numbers, gate statuses, figures, and claim boundaries to
  their audited source files;
- `results/figs/`: deterministic PDF/PNG main figures, captions, and a source
  hash manifest, including the untouched D4 ResNet-20, oral-B-v2, and
  60-record standard-depth confirmations, plus audited RRM failure and
  dynamic-stability supplements.

The current main figures show the local-method Pareto frontier, credit
assignment versus depth, the load-bearing innovation ablation, and the
untouched standard-ResNet confirmations, including positive ResNet-20-to-56
scaling. The first three require exactly seeds 0--4; the ResNet panels require
paired test seeds 10--14. Strict renderers refuse missing, dirty,
protocol-mixed, source-drifted, or gate-inconsistent cells.

## Verification

The lab environment currently used for audited CPU checks is
`/home/yurenh2/miniconda3/envs/ep_pascal`. Run:

```bash
experiments/finalize_accept.sh
```

This regenerates all publication-facing figures and manifests, enforces the
frozen Pareto/scaling, D4, oral-B-v2, and 60-record standard-depth gates,
verifies baseline and protocol mechanics, checks the local-rule smoke tests,
evaluates the theory identities, and rebuilds the audited tables.

The convolutional infrastructure can be checked independently:

```bash
/home/yurenh2/miniconda3/envs/ep_pascal/bin/python3 experiments/cifar_image_smoke.py
/home/yurenh2/miniconda3/envs/ep_pascal/bin/python3 experiments/conv_local_smoke.py
```

These checks cover deterministic CIFAR splitting/augmentation, the standard
CIFAR `6n+2` ResNet topology, exact local convolution and BatchNorm
eligibilities, the BatchNorm-coupled perturbation objective, predictor fitting,
translation-shared feedback, representable-subspace causal calibration, and
parameter/cost accounting.  The latter is a transparent post-A3 development
path: it perturbs the channel-gated basis directly and reproduces the expected
full-field delta rule with far less irrelevant spatial variance.  It does not
retroactively reopen or relabel the failed frozen Oral-A protocol.

The same smoke suite also audits a convolutional hierarchical-FA baseline. Its
random feedback tensors follow the real residual DAG and use local activation
Jacobians, but remain independent of forward weights. An audit-only symmetric
copy reproduces exact hidden gradients to `5.3e-16` relative error and the BP
parameter update to `2.98e-8`; actual HFA never performs that copy.
`HFA_BASELINE.md` freezes its matched ResNet-20 validation screen before any
HFA accuracy endpoint is observed.

The learned-hierarchy development path perturbs the feedback convolution
parameter spaces themselves. One antithetic pair estimates every edge's causal
target moment simultaneously, while the current prediction moment is a local
child/parent correlation. Its BatchNorm-coupled JVP and symmetric-limit delta
rule are executable smoke tests. `ORAL_A_V4.md` freezes the causal-capture and
conditional accuracy gates; no accuracy claim is attached until they pass.

Because the global-scalar V4 gate failed, the next strong inherited comparator
uses normalized local response mirroring. Its observation phase sends Gaussian
parent activity through ordinary local forward synapses; a separate feedback
update sees only the probe/child-response pairs. The smoke suite verifies both
high finite-sample kernel recovery and that changing forward parameters after
observation cannot affect the update. Any scale obtained this way is credited
to the weight-mirror baseline until innovation residualization is shown to be
load-bearing on top of it.
`MIRROR_BASELINE.md` freezes that comparator's CIFAR-prefix capture, short
accuracy, and conditional full-validation gates.

After estimate-then-average WM narrowly failed its short accuracy gate, the
next substantive inherited control uses residual response mirroring. Q predicts
the observed child response and learns only from the local prediction error;
unlike WM, its update is zero for every probe at exact symmetry. The extra
feedback prediction convolution is charged explicitly, and this rule receives
no SDIL novelty credit.
`RESIDUAL_MIRROR.md` freezes its capture and conditional task gates.

The stronger reciprocal Kolen--Pollack substrate is audited separately in
`KP_BASELINE.md`. Its short and full frozen gates pass; the full record reaches
91.26% validation accuracy, 0.9994 early alignment, and 1.326x BP MACs without
feedback loss queries. Before that full endpoint, `MIXED_TRAFFIC.md` froze the
actual Harnett-specific test: raw, norm-matched raw, and innovation under
identical four-times-RMS soma-predictable apical traffic and predictor cost.
Mechanics pass, but the complete MT-1 panel fails: all three signals become
nonfinite in epoch 1 and end at 10.00% validation accuracy. MT-2 and MT-3
remain untouched, closing this standard-ResNet recovery without a weaker
traffic intervention.

The subsequent V3 mechanism estimates the required A/G matrix statistics
directly by perturbing the vectorizer parameter subspace. It remains
forward-only and uses two causal loss queries per direction. Its exact and
BatchNorm JVP identities, delta rule, and matched synthetic variance comparison
are covered by `experiments/conv_local_smoke.py`; no task-level claim is made
until a new frozen gate is completed.

`experiments/finalize_accept.sh` additionally requires strict imports of the
frozen BurstCCN and Dual Propagation author-code runs. Both records now pass;
the accept-bar mechanical audit is green. The original standard-ResNet branch
stopped at its failed A3 validation gate and its A4 panel remains sealed; the
independently authorized v2 recovery is the separate passed 60-record panel.

## Result discipline

Every publication-facing run records the source revision, tracked-tree state,
full arguments, split identity, seed, evaluation count, wall time, query/MAC
accounting, and final finiteness. Development, validation, and untouched
confirmation results are never pooled. Failed gates close their branch instead
of triggering seed deletion or post-hoc threshold changes.

The current formal milestone is `9/10` after the untouched D4, oral-B-v2, and
standard-depth confirmations. A conservative external-review forecast is
`8/10`: the full matched nine-method crossover, original-data biological
validation, and a credit pathway novel beyond inherited perturbation/KP
mechanisms remain open. Scores change only after an audited frozen stage, not
after a pilot or presentation improvement.