path: root/README.md

# Somato-Dendritic Innovation Learning (SDIL)

SDIL is a non-backpropagation learning project motivated by the neuron-specific
somato-dendritic residuals reported by Harnett and colleagues. Its central
algorithmic hypothesis is narrow: a mixed apical compartment should teach from
the component that is unexpected given the same neuron's ordinary somatic
state, rather than from raw apical activity.

For hidden population `l`, the implemented rule is

```text
a_l       = A_l c + ordinary apical traffic
a_hat_l   = P_l(h_l), fitted during neutral periods
r_l       = a_l - a_hat_l
Delta W_l = eta (r_l * local postsynaptic gain) h_(l-1)^T.
```

`A_l` is calibrated from antithetic node perturbations without reverse-mode
differentiation or weight transport. That learned-feedback mechanism is
inherited from Lansdell, Prakash, and Kording (2020); it is not claimed as
novel. The candidate contribution is the neutral-period, per-cell innovation
operation under mixed apical traffic, together with its theory, causal audit,
and scaling behavior. See `NOVELTY.md` for the exact prior-art boundary.

## Current audited evidence

- On flattened CIFAR-10, SDIL loses only `0.214 +/- 0.349` accuracy points from
  hidden depth 5 to 60, while DFA's early-layer teaching alignment falls from
  `0.514` to `0.047`. This is depth preservation on a depth-flat task, not yet
  evidence that added depth is useful.
- With strong soma-predictable apical traffic, raw and norm-matched raw
  learning fall to `10.38%` and `10.31%`, while innovation learning retains
  `97.35%`. The paired innovation gain over norm-matched raw is
  `87.038 +/- 0.607` points across five seeds.
- Direct, unamortized node perturbation passes the controlled useful-depth
  diagnosis, but costs `68.4x` ordinary forward-equivalent work. The learned
  apical vectorizer, rather than the local eligibility rule, is the current
  useful-depth bottleneck.
- A low-query K1/every-4 calibration protocol retains `112.9%` of the K16 gain
  over DFA with 16x fewer logical queries, 11x less calibration work, and 5.3x
  less total forward-equivalent work in the frozen MLP protocol.
- The frozen standard-ResNet funnel passed its BP reference and short screen:
  BP reached `91.62%`, and short-run channel-gated SDIL reached `41.98%`
  versus tuned DFA's `37.16%`. The full A3 run then failed decisively: SDIL
  became nonfinite at epoch 89 and ended at chance, while DFA remained finite
  at `33.06%`. A4 was therefore not opened and no confirmation test seed was
  touched.
- The post-failure representable-subspace estimator passed its exact mechanics
  tests and improved frozen-forward early/all-layer alignment from
  `0.0011/0.0117` to `0.0072/0.0527` at matched query count. It nevertheless
  missed the frozen `0.01` early-layer and improvement gates, so no v2 full
  ResNet run was launched.
- Direct vectorizer-space perturbation then passed its exact mechanics and
  synthetic variance audit, but V3-1 again failed: early alignment was
  `0.00714` versus matched V2's `0.00721`, although all-layer alignment rose to
  `0.06258`. V3 full training therefore remained closed.
- Native author-code fidelity is complete. BurstCCN reaches `80.10%` at its
  validation-selected epoch versus published `82.97 +/- 0.21%`; Dual Prop
  reaches `92.46%` versus published `92.41 +/- 0.07%`. Their audited walls are
  `15451.3 s` and `23119.8 s`, respectively, and they are not presented as
  matched-compute points.

The broad endogenous-traffic gate and the Harnett desired-velocity/online-control
screen failed. Those results are retained and explicitly constrain the paper:
SDIL does not currently explain the reported error-derivative signature, and
the residual mechanism is supported for traffic predictable from the chosen
somatic statistic rather than arbitrary top-down context. `ROADMAP.md` and
`ORAL_B.md` contain the frozen gates and complete negative branches.

## Publication-facing artifacts

- `RESULTS.md`: audited positive and negative results;
- `THEORY.md`: estimator variance, descent conditions, conditional innovation,
  timescales, and hardware-independent cost;
- `BASELINES.md`: matched and native-author baseline ledger;
- `ROADMAP.md`: accept/oral evidence gates and their state;
- `ORAL_A.md`: frozen standard CIFAR ResNet funnel;
- `ORAL_A_V2.md`: frozen post-failure representable-subspace funnel;
- `ORAL_A_V3.md`: frozen vectorizer-space causal-calibration funnel;
- `REVIEW_SCORECARD.md`: adversarial ICLR-style score trajectory;
- `results/figs/`: deterministic PDF/PNG main figures, captions, and a source
  hash manifest.

The three current main figures show the local-method Pareto frontier, credit
assignment versus depth, and the load-bearing innovation ablation. Every final
cell has exactly seeds 0--4 and clean git provenance; the strict renderer
refuses missing or protocol-mixed cells.

## Verification

The lab environment currently used for audited CPU checks is
`/home/yurenh2/miniconda3/envs/ep_pascal`. Run:

```bash
experiments/finalize_claims.sh
```

This regenerates the figures and manifest, enforces the frozen Pareto/scaling
gates, verifies baseline and protocol mechanics, checks the local-rule smoke
tests, evaluates the theory identities, and rebuilds the audited tables.

The convolutional infrastructure can be checked independently:

```bash
/home/yurenh2/miniconda3/envs/ep_pascal/bin/python3 experiments/cifar_image_smoke.py
/home/yurenh2/miniconda3/envs/ep_pascal/bin/python3 experiments/conv_local_smoke.py
```

These checks cover deterministic CIFAR splitting/augmentation, the standard
CIFAR `6n+2` ResNet topology, exact local convolution and BatchNorm
eligibilities, the BatchNorm-coupled perturbation objective, predictor fitting,
translation-shared feedback, representable-subspace causal calibration, and
parameter/cost accounting.  The latter is a transparent post-A3 development
path: it perturbs the channel-gated basis directly and reproduces the expected
full-field delta rule with far less irrelevant spatial variance.  It does not
retroactively reopen or relabel the failed frozen Oral-A protocol.

The same smoke suite also audits a convolutional hierarchical-FA baseline. Its
random feedback tensors follow the real residual DAG and use local activation
Jacobians, but remain independent of forward weights. An audit-only symmetric
copy reproduces exact hidden gradients to `5.3e-16` relative error and the BP
parameter update to `2.98e-8`; actual HFA never performs that copy.
`HFA_BASELINE.md` freezes its matched ResNet-20 validation screen before any
HFA accuracy endpoint is observed.

The learned-hierarchy development path perturbs the feedback convolution
parameter spaces themselves. One antithetic pair estimates every edge's causal
target moment simultaneously, while the current prediction moment is a local
child/parent correlation. Its BatchNorm-coupled JVP and symmetric-limit delta
rule are executable smoke tests. `ORAL_A_V4.md` freezes the causal-capture and
conditional accuracy gates; no accuracy claim is attached until they pass.

Because the global-scalar V4 gate failed, the next strong inherited comparator
uses normalized local response mirroring. Its observation phase sends Gaussian
parent activity through ordinary local forward synapses; a separate feedback
update sees only the probe/child-response pairs. The smoke suite verifies both
high finite-sample kernel recovery and that changing forward parameters after
observation cannot affect the update. Any scale obtained this way is credited
to the weight-mirror baseline until innovation residualization is shown to be
load-bearing on top of it.
`MIRROR_BASELINE.md` freezes that comparator's CIFAR-prefix capture, short
accuracy, and conditional full-validation gates.

After estimate-then-average WM narrowly failed its short accuracy gate, the
next substantive inherited control uses residual response mirroring. Q predicts
the observed child response and learns only from the local prediction error;
unlike WM, its update is zero for every probe at exact symmetry. The extra
feedback prediction convolution is charged explicitly, and this rule receives
no SDIL novelty credit.
`RESIDUAL_MIRROR.md` freezes its capture and conditional task gates.

The subsequent V3 mechanism estimates the required A/G matrix statistics
directly by perturbing the vectorizer parameter subspace. It remains
forward-only and uses two causal loss queries per direction. Its exact and
BatchNorm JVP identities, delta rule, and matched synthetic variance comparison
are covered by `experiments/conv_local_smoke.py`; no task-level claim is made
until a new frozen gate is completed.

`experiments/finalize_accept.sh` additionally requires strict imports of the
frozen BurstCCN and Dual Propagation author-code runs. Both records now pass;
the accept-bar mechanical audit is green. The standard ResNet branch stopped
at its failed A3 validation gate, and the untouched A4 test panel remains
sealed.

## Result discipline

Every publication-facing run records the source revision, tracked-tree state,
full arguments, split identity, seed, evaluation count, wall time, query/MAC
accounting, and final finiteness. Development, validation, and untouched
confirmation results are never pooled. Failed gates close their branch instead
of triggering seed deletion or post-hoc threshold changes.

The current strict reviewer estimate is `5/10` (borderline reject, confidence
`4/5`): the mechanism and controlled depth-preservation results are strong,
but the frozen standard-useful-scale attempt failed. The score changes only
after an audited frozen stage, not after a pilot or a presentation improvement.