path: root/ROADMAP.md

# SDIL ICLR 2027 evidence roadmap

This document predeclares the evidence gates used to decide which claims survive into a paper.
It is intentionally stricter than a list of experiments: a failed gate narrows the claim rather
than triggering selective seed removal or post-hoc protocol changes.

## Order of work

1. **Accept bar:** establish a load-bearing innovation mechanism, useful-depth scaling, fair cost,
   theory, and the closest baselines.
2. **Oral bar B:** connect the mechanism to the signatures and new predictions motivated by
   Harnett et al.'s somato-dendritic residuals.
3. **Oral bar A:** demonstrate the same mechanism in standard CNN/ResNet families. The runner is
   developed early so long jobs can use otherwise idle, explicitly authorized GPUs without
   delaying stages 1–2.

## Frozen accept claims and gates

### C1. Innovation is necessary under naturally mixed apical traffic

**Status: failed the frozen MNIST confirmation on 2026-07-22.** Three of four traffic
realizations passed, but top-down projection seed 1234 produced a `+1.520`-point innovation gain
over norm-matched raw feedback, below the predeclared `+2`-point threshold. The other top-down
projection gained `+2.956` points and the two soma-traffic projections gained `+2.860` and
`+3.180` points. All four passed the traffic-R2 and alignment requirements, and the no-traffic
control had exactly zero mean change. The failed result is retained; it cannot be repaired by
changing the MNIST threshold or excluding the projection. A new dataset may receive a separately
frozen validation-only selection followed by an untouched test confirmation.

The apical compartment must contain ordinary non-teaching traffic generated by the model or task,
not only an additive nuisance chosen to match the predictor. Candidate traffic includes
back-propagating somatic activity, top-down contextual state, and task-irrelevant feedback shared
between neutral and teaching periods.

Required controls use the same forward architecture, initialization, data order, and update norm:

- raw apical signal;
- raw apical signal matched per sample to the innovation norm;
- somato-dendritic innovation;
- innovation with the predictor trained during task periods rather than neutral periods;
- no-traffic control, where raw and innovation should agree.

**Gate C1:** across at least two independently generated traffic families, innovation must improve
held-out accuracy by at least 2 percentage points over norm-matched raw feedback at a predeclared
traffic level, without harming the no-traffic setting by more than 0.5 points. The predictor must
remove measurable soma-predictable traffic while retaining causal teaching alignment.

The validation-only timescale screen froze `eta_P=0.05` and 200 neutral warmup steps by the highest
mean last-epoch accuracy across top-down traffic scales 0.2 and 0.5 (and the best worst-scale
endpoint). C1 must use this choice without further test-set tuning. Traffic projection seeds remain
separate from model/data seeds and must be recorded explicitly.

The frozen confirmation is MNIST d3/w256 for 15 epochs with the C3-selected low-query calibration
(`K=1`, every four minibatches). It crosses five model seeds with soma traffic at `rho=0.5` and
top-down traffic at `rho=0.2`, each using independently generated projection seeds 1234 and 5678.
Every traffic realization crosses raw, norm-matched raw, neutral-period innovation, and task-period
predictor controls; a no-traffic panel crosses the first three. This is 95 runs total. Test accuracy
is evaluated once after training, while final causal-alignment diagnostics use a fixed 512-example
training prefix. C1 passes only if every one of the four traffic realizations has a mean paired
innovation gain of at least 2 points over norm-matched raw, mean traffic R2 of at least 0.5, and
mean final hidden-layer teaching alignment of at least 0.1. The mean paired no-traffic innovation
change relative to both raw controls must be no worse than -0.5 points.

After the MNIST near-miss, one independent FashionMNIST recovery is frozen before any new result.
The validation screen uses model seed 0, a 5,000-example stratified split from training data
(`split_seed=2027`), d3/w256, 15 epochs, and otherwise the exact MNIST protocol. It crosses soma
traffic at `rho=0.5` and top-down traffic at `rho in {0.2, 0.5}`, each with projection seeds 1234
and 5678 and all four signal controls; a no-traffic panel has three controls (27 runs total).
Only final validation accuracy is observed; diagnostics use the fixed training prefix.

Soma traffic is eligible only if both projections gain at least 2 points over norm-matched raw,
retain accuracy within 3 points of no traffic, have traffic R2 at least 0.5, and alignment at least
0.1. A top-down level is eligible only if both its projections meet the same criteria. Among
eligible levels, select the one with the highest worst-projection gain, breaking an exact tie
toward smaller `rho`. No-traffic harm must remain within 0.5 points. If soma or no top-down level
is eligible, stop without evaluating FashionMNIST test. Otherwise freeze that level for a
five-model-seed, 95-run test confirmation under the original C1 per-realization gate.

**FashionMNIST recovery status: failed validation on 2026-07-22; test remains unevaluated.** Soma
traffic passed strongly in both projections. Neither top-down level was eligible: at `rho=0.2`,
projection 1234 gained only `+1.300` points and both residual endpoints were more than 3 points
below no traffic; at `rho=0.5`, both gains were large but both residual endpoints were more than
10 points below no traffic. This closes the predeclared recovery branch. C1 remains failed, and
the supported mechanism claim is narrowed to removal of soma-predictable ordinary traffic rather
than arbitrary contextual apical traffic.

### C2. SDIL scales when depth is useful

Flattened CIFAR is retained as a preservation result, not as evidence that added depth is used.
The primary controlled task must make BP accuracy increase with depth. Post-training block lesions
and residual-branch statistics must verify that the deep model actually uses its extra blocks.

**Gate C2:** BP must gain at least 5 points from the shallow to the deep configuration. SDIL must
recover at least 70% of that paired gain; the strongest competing local rule must recover no more
than 50%, or SDIL must beat it by at least 2 points at the deep endpoint. Removing the final third
of trained blocks must reduce SDIL performance, ruling out an effective shallow-network solution.

The C2 candidate is frozen from scratch pilots before formal validation: a two-level Telgarsky
tent-map binary task, one informative input, width 8, ReLU residual students, 80 epochs,
batch size 256, `eta=0.03`, and momentum 0.9. Each task has 10,000 generated training examples,
of which a stratified 2,000-example validation split (`split_seed=2027`) is held out; its untouched
test set has 5,000 independently generated examples. Shallow/deep are fixed to d1/d4. A BP shape
screen uses task seeds 0, 1, 2, student seed 0, and depths 1, 2, 3, 4, 6. Contextual depths cannot
replace d4 after results are observed. The BP screen passes only if mean d1-to-d4 gain is at least
5 points, every task seed gains, mean d4 final-third-lesion drop is at least 2 points, and every
task seed is hurt by the lesion. Only final validation metrics are observed.

**BP useful-depth status: passed on 2026-07-22.** Across task seeds 0--2, d1-to-d4 gains were
`+22.10`, `+42.85`, and `+33.90` points (mean `+32.95`). Bypassing the final third of d4 blocks
reduced accuracy by `40.85`, `35.05`, and `34.85` points (mean `36.92`). The contextual d6 curve
was less stable, so the preselected d4 endpoint remains fixed.

If BP passes, the local-rule validation panel freezes d1/d4 for BP, FA, DFA, and no-traffic SDIL
with C3's `K=1/every=4`, crossing the same three task seeds with student seeds 0--4. Its test panel
may run only if the original C2 gate holds on validation and the d4 SDIL lesion loses at least
2 points on average with at least 10 of 15 paired runs harmed. Test confirmation retrains on the
same 8,000-example training subset and evaluates each task seed's untouched 5,000 examples once.

**Local-rule validation status: failed on 2026-07-22; test remains unevaluated.** Across the 15
task/model pairs, BP gained `+29.697` points from d1 to d4 while SDIL gained only `+6.240`
(`21.0%` recovery). FA and DFA recovered `62.5%` and `59.5%`; SDIL's d4 endpoint was `5.560`
points below the stronger comparator. The SDIL lesion criterion itself passed (`+12.787`-point
mean drop; 11/15 harmed), so the failure is credit/optimization rather than unused depth. The
fixed linear output-error vectorizer `A_l c` is now the leading limitation: increasing perturbation
directions/frequency or simple gain normalization did not stabilize scratch validation. No C2
test panel may run unless a separately declared mechanism change passes validation.

The mechanism-recovery candidate is a bounded context-conditioned vectorizer,
`a_l = A_l c + G_l vec(c outer tanh(h_top))`, with both pathways calibrated by the same local
node-perturbation regression. The gate term is zero-initialized and vanishes when `c=0`, preserving
neutral-period innovation identification. An exploratory task-seed-0 validation budget compared
linear, per-cell soma-gated, and context-gated forms; `eta_A in {0.005, 0.01, 0.02}`; and K1/e2,
K1/e4, and K4/e4. Bounded context gating with `eta_A=0.01`, K1/e4 was selected by mean d4
validation accuracy. It recovered 70.2% of BP's task-0 depth gain but retained one bad basin.

Before any C2 test, an independent confirmation is frozen on new task seeds 3, 4, 5 and student
seeds 0--4, crossing BP, FA, DFA, and context-SDIL at d1/d4 (120 validation-only runs). It uses the
same original C2 thresholds, and additionally requires at least 10/15 context-SDIL depth gains to
be positive; its d4 lesion must lose at least 2 points on average and harm at least 10/15 runs.
Failure closes this recovery. Passing freezes a single test confirmation over task seeds 0--5,
student seeds 0--4, all four methods, and d1/d4, retrained on the unchanged 8,000-example subsets.

### C3. Causal calibration is genuinely amortized

**Status: passed on 2026-07-22.** The frozen five-seed confirmation retained 112.9% of the K16/e4
gain over DFA while using 16x fewer perturbation loss queries, 11x less calibration
forward-equivalent work, and 5.3x less total training forward-equivalent work. Peak allocated
memory fell from 803.5 MiB to 739.8 MiB. These numbers apply to the audited CIFAR-10 d20/w64
protocol; extrapolation to standard CNNs remains an oral-A question.

Report wall time, forward-equivalent work, number of scalar loss queries, peak device memory, and
the perturbation batch expansion. Sweep perturbation frequency and number of directions under both
fixed epoch and fixed query/FLOP budgets.

**Gate C3:** a protocol using at least 10x fewer perturbation loss queries than the current
16-directions-every-4-steps setting must preserve at least 90% of its gain over DFA. Pareto claims
must hold in a hardware-independent cost coordinate as well as GTX-1080 wall time.

The validation screen is frozen to CIFAR-10 d20/w64, five epochs, seed 0, and compares the current
`K=16/every=4` reference with `K/every` pairs `1/4`, `2/8`, `4/16`, `1/8`, `1/16`, and `2/32`, plus
DFA. Selection uses final validation accuracy and logical batch loss queries; the test split is not
evaluated. `K=1/every=4` was selected: it used 16x fewer batch loss queries and retained 177.5% of
the reference gain over DFA. This selection is now frozen; the protocol receives a five-seed test
confirmation with no further tuning before C3 can be declared passed.

### C4. The comparison set contains the closest alternatives

The main comparison must include exact BP, FA, DFA, direct/unamortized node perturbation, learned
direct node-perturbation feedback (Lansdell et al. 2020), Dual Prop, and BurstCCN. The Lansdell
method is the exact no-traffic/P=0 backbone of the current implementation, not merely a related
baseline. BurstCCN is already demonstrated on CIFAR-10 and ImageNet and already models the Harnett
BCI signatures, so it is the strongest dendritic/scaling comparator. EP remains an informative
relaxation-based baseline. Forward-Forward and PEPITA are appendix context unless they become
competitive under the frozen protocol.

Each method receives a documented validation budget. Native-protocol and exact-architecture
comparisons are labelled separately; neither substitutes for a matched compute/query comparison.

### C5. Theory predicts the observed regimes

The minimum theory package contains:

1. bias and variance of simultaneous multi-layer perturbation, including cross-layer interference;
2. a smooth-loss descent bound separating angular alignment, gain calibration, and curvature;
3. an innovation/SNR result for subtracting soma-predictable apical traffic;
4. an explicit complexity table covering learning phases, transport, loss queries, FLOPs, and
   memory.

Numerical simulations must test the predicted scaling with depth, width, directions, perturbation
scale, and predictor timescale.

## Protocol discipline

- Hyperparameters are selected on a validation split created only from the training set.
- The test set is evaluated for frozen candidate protocols, never used to choose schedules.
- Main results use five model seeds. Synthetic task claims additionally vary the task/teacher seed.
- Every completed finite trajectory is retained; crashes and excluded runs are reported with a
  reason fixed before inspecting accuracy.
- Pilot results remain versioned but cannot be pooled with frozen runs.
- Every result records source revision, dirty state, protocol identifier, data split hash, logical
  loss queries, forward-equivalent work, peak memory, and wall time.

## Oral bar B: biological bridge

After C1–C5 pass, test whether the model reproduces the qualitative Harnett signatures:

- dendritic activity contains information absent after conditioning on somatic activity;
- the residual decodes outcome/reward-related events;
- residual sign follows the neuron's causal contribution to the objective;
- residual predicts subsequent activity change or desired velocity;
- selectively disrupting the residual impairs learning more than matched nonspecific disruption.

The paper needs at least one prediction not used to construct SDIL. A preferred route is to request
the original data/analysis from the authors and test the prediction out of sample; otherwise the
claim is explicitly computational rather than a fit to cortical data.

## Oral bar A: standard deep architectures

Prepare convolutional local-update primitives and ResNet-20/32/56 protocols early. Queue frozen
runs opportunistically on authorized idle GPUs. Because BurstCCN already reports CIFAR-10 and
ImageNet scaling, dataset scale alone is not novel. The oral-level target is a memorable joint
result: near-BP accuracy in a standard architecture, a win over or materially simpler/lower-cost
regime than BurstCCN, and a nondominated accuracy–hardware-independent-cost point. Scale alone is
also insufficient if C1 does not show that somato-dendritic innovation is load-bearing.

## Stop conditions

- Do not extend flattened-CIFAR MLP depth merely to obtain a larger depth number.
- Do not add MNIST EP seeds unless a protocol bug invalidates the completed five-seed panel.
- Do not promote a wall-time frontier that disappears under loss-query or FLOP accounting.
- Do not describe the algorithm as a cortical implementation if simultaneous perturbation or
  supervision assumptions remain biologically unsupported.