path: root/ROADMAP.md

# SDIL ICLR 2027 evidence roadmap

This document predeclares the evidence gates used to decide which claims survive into a paper.
It is intentionally stricter than a list of experiments: a failed gate narrows the claim rather
than triggering selective seed removal or post-hoc protocol changes.

## Order of work

1. **Accept bar:** establish a load-bearing innovation mechanism, useful-depth scaling, fair cost,
   theory, and the closest baselines.
2. **Oral bar B:** connect the mechanism to the signatures and new predictions motivated by
   Harnett et al.'s somato-dendritic residuals.
3. **Oral bar A:** demonstrate the same mechanism in standard CNN/ResNet families. The runner is
   developed early so long jobs can use otherwise idle, explicitly authorized GPUs without
   delaying stages 1–2.

## Frozen accept claims and gates

### C1. Innovation is necessary under naturally mixed apical traffic

**Status: failed the frozen MNIST confirmation on 2026-07-22.** Three of four traffic
realizations passed, but top-down projection seed 1234 produced a `+1.520`-point innovation gain
over norm-matched raw feedback, below the predeclared `+2`-point threshold. The other top-down
projection gained `+2.956` points and the two soma-traffic projections gained `+2.860` and
`+3.180` points. All four passed the traffic-R2 and alignment requirements, and the no-traffic
control had exactly zero mean change. The failed result is retained; it cannot be repaired by
changing the MNIST threshold or excluding the projection. A new dataset may receive a separately
frozen validation-only selection followed by an untouched test confirmation.

The apical compartment must contain ordinary non-teaching traffic generated by the model or task,
not only an additive nuisance chosen to match the predictor. Candidate traffic includes
back-propagating somatic activity, top-down contextual state, and task-irrelevant feedback shared
between neutral and teaching periods.

Required controls use the same forward architecture, initialization, data order, and update norm:

- raw apical signal;
- raw apical signal matched per sample to the innovation norm;
- somato-dendritic innovation;
- innovation with the predictor trained during task periods rather than neutral periods;
- no-traffic control, where raw and innovation should agree.

**Gate C1:** across at least two independently generated traffic families, innovation must improve
held-out accuracy by at least 2 percentage points over norm-matched raw feedback at a predeclared
traffic level, without harming the no-traffic setting by more than 0.5 points. The predictor must
remove measurable soma-predictable traffic while retaining causal teaching alignment.

The validation-only timescale screen froze `eta_P=0.05` and 200 neutral warmup steps by the highest
mean last-epoch accuracy across top-down traffic scales 0.2 and 0.5 (and the best worst-scale
endpoint). C1 must use this choice without further test-set tuning. Traffic projection seeds remain
separate from model/data seeds and must be recorded explicitly.

The frozen confirmation is MNIST d3/w256 for 15 epochs with the C3-selected low-query calibration
(`K=1`, every four minibatches). It crosses five model seeds with soma traffic at `rho=0.5` and
top-down traffic at `rho=0.2`, each using independently generated projection seeds 1234 and 5678.
Every traffic realization crosses raw, norm-matched raw, neutral-period innovation, and task-period
predictor controls; a no-traffic panel crosses the first three. This is 95 runs total. Test accuracy
is evaluated once after training, while final causal-alignment diagnostics use a fixed 512-example
training prefix. C1 passes only if every one of the four traffic realizations has a mean paired
innovation gain of at least 2 points over norm-matched raw, mean traffic R2 of at least 0.5, and
mean final hidden-layer teaching alignment of at least 0.1. The mean paired no-traffic innovation
change relative to both raw controls must be no worse than -0.5 points.

After the MNIST near-miss, one independent FashionMNIST recovery is frozen before any new result.
The validation screen uses model seed 0, a 5,000-example stratified split from training data
(`split_seed=2027`), d3/w256, 15 epochs, and otherwise the exact MNIST protocol. It crosses soma
traffic at `rho=0.5` and top-down traffic at `rho in {0.2, 0.5}`, each with projection seeds 1234
and 5678 and all four signal controls; a no-traffic panel has three controls (27 runs total).
Only final validation accuracy is observed; diagnostics use the fixed training prefix.

Soma traffic is eligible only if both projections gain at least 2 points over norm-matched raw,
retain accuracy within 3 points of no traffic, have traffic R2 at least 0.5, and alignment at least
0.1. A top-down level is eligible only if both its projections meet the same criteria. Among
eligible levels, select the one with the highest worst-projection gain, breaking an exact tie
toward smaller `rho`. No-traffic harm must remain within 0.5 points. If soma or no top-down level
is eligible, stop without evaluating FashionMNIST test. Otherwise freeze that level for a
five-model-seed, 95-run test confirmation under the original C1 per-realization gate.

### C2. SDIL scales when depth is useful

Flattened CIFAR is retained as a preservation result, not as evidence that added depth is used.
The primary controlled task must make BP accuracy increase with depth. Post-training block lesions
and residual-branch statistics must verify that the deep model actually uses its extra blocks.

**Gate C2:** BP must gain at least 5 points from the shallow to the deep configuration. SDIL must
recover at least 70% of that paired gain; the strongest competing local rule must recover no more
than 50%, or SDIL must beat it by at least 2 points at the deep endpoint. Removing the final third
of trained blocks must reduce SDIL performance, ruling out an effective shallow-network solution.

### C3. Causal calibration is genuinely amortized

**Status: passed on 2026-07-22.** The frozen five-seed confirmation retained 112.9% of the K16/e4
gain over DFA while using 16x fewer perturbation loss queries, 11x less calibration
forward-equivalent work, and 5.3x less total training forward-equivalent work. Peak allocated
memory fell from 803.5 MiB to 739.8 MiB. These numbers apply to the audited CIFAR-10 d20/w64
protocol; extrapolation to standard CNNs remains an oral-A question.

Report wall time, forward-equivalent work, number of scalar loss queries, peak device memory, and
the perturbation batch expansion. Sweep perturbation frequency and number of directions under both
fixed epoch and fixed query/FLOP budgets.

**Gate C3:** a protocol using at least 10x fewer perturbation loss queries than the current
16-directions-every-4-steps setting must preserve at least 90% of its gain over DFA. Pareto claims
must hold in a hardware-independent cost coordinate as well as GTX-1080 wall time.

The validation screen is frozen to CIFAR-10 d20/w64, five epochs, seed 0, and compares the current
`K=16/every=4` reference with `K/every` pairs `1/4`, `2/8`, `4/16`, `1/8`, `1/16`, and `2/32`, plus
DFA. Selection uses final validation accuracy and logical batch loss queries; the test split is not
evaluated. `K=1/every=4` was selected: it used 16x fewer batch loss queries and retained 177.5% of
the reference gain over DFA. This selection is now frozen; the protocol receives a five-seed test
confirmation with no further tuning before C3 can be declared passed.

### C4. The comparison set contains the closest alternatives

The main comparison must include exact BP, FA, DFA, direct/unamortized node perturbation, learned
direct node-perturbation feedback (Lansdell et al. 2020), Dual Prop, and BurstCCN. The Lansdell
method is the exact no-traffic/P=0 backbone of the current implementation, not merely a related
baseline. BurstCCN is already demonstrated on CIFAR-10 and ImageNet and already models the Harnett
BCI signatures, so it is the strongest dendritic/scaling comparator. EP remains an informative
relaxation-based baseline. Forward-Forward and PEPITA are appendix context unless they become
competitive under the frozen protocol.

Each method receives a documented validation budget. Native-protocol and exact-architecture
comparisons are labelled separately; neither substitutes for a matched compute/query comparison.

### C5. Theory predicts the observed regimes

The minimum theory package contains:

1. bias and variance of simultaneous multi-layer perturbation, including cross-layer interference;
2. a smooth-loss descent bound separating angular alignment, gain calibration, and curvature;
3. an innovation/SNR result for subtracting soma-predictable apical traffic;
4. an explicit complexity table covering learning phases, transport, loss queries, FLOPs, and
   memory.

Numerical simulations must test the predicted scaling with depth, width, directions, perturbation
scale, and predictor timescale.

## Protocol discipline

- Hyperparameters are selected on a validation split created only from the training set.
- The test set is evaluated for frozen candidate protocols, never used to choose schedules.
- Main results use five model seeds. Synthetic task claims additionally vary the task/teacher seed.
- Every completed finite trajectory is retained; crashes and excluded runs are reported with a
  reason fixed before inspecting accuracy.
- Pilot results remain versioned but cannot be pooled with frozen runs.
- Every result records source revision, dirty state, protocol identifier, data split hash, logical
  loss queries, forward-equivalent work, peak memory, and wall time.

## Oral bar B: biological bridge

After C1–C5 pass, test whether the model reproduces the qualitative Harnett signatures:

- dendritic activity contains information absent after conditioning on somatic activity;
- the residual decodes outcome/reward-related events;
- residual sign follows the neuron's causal contribution to the objective;
- residual predicts subsequent activity change or desired velocity;
- selectively disrupting the residual impairs learning more than matched nonspecific disruption.

The paper needs at least one prediction not used to construct SDIL. A preferred route is to request
the original data/analysis from the authors and test the prediction out of sample; otherwise the
claim is explicitly computational rather than a fit to cortical data.

## Oral bar A: standard deep architectures

Prepare convolutional local-update primitives and ResNet-20/32/56 protocols early. Queue frozen
runs opportunistically on authorized idle GPUs. Because BurstCCN already reports CIFAR-10 and
ImageNet scaling, dataset scale alone is not novel. The oral-level target is a memorable joint
result: near-BP accuracy in a standard architecture, a win over or materially simpler/lower-cost
regime than BurstCCN, and a nondominated accuracy–hardware-independent-cost point. Scale alone is
also insufficient if C1 does not show that somato-dendritic innovation is load-bearing.

## Stop conditions

- Do not extend flattened-CIFAR MLP depth merely to obtain a larger depth number.
- Do not add MNIST EP seeds unless a protocol bug invalidates the completed five-seed panel.
- Do not promote a wall-time frontier that disappears under loss-query or FLOP accounting.
- Do not describe the algorithm as a cortical implementation if simultaneous perturbation or
  supervision assumptions remain biologically unsupported.