path: root/RESULTS.md

# SDIL — results log (audited sections report n; timan107 GTX-1080 / ep_pascal)

## 2026-07-21 audit and version boundary

Git was initialized after inheriting the project. The original code/results are preserved at
`215acff`. The following audit findings changed the canonical implementation:

1. `49b86d6`: residual-block local updates now include the branch Jacobian
   `res_alpha=1/sqrt(depth)`. The omission preserved cosine direction but inflated update norms by
   `sqrt(depth)`, confounding cross-depth optimization comparisons.
2. `5255166`: the Harnett baseline is now a per-neuron affine soma-dendrite fit
   `ahat_l = p_l * h_l + b_l`, rather than a dense population map. This follows the paper's
   per-cell least-squares somatic-event to dendritic-event relation.
3. `c2769e6`: causal perturbation targets and predicted residuals are measured on the same
   pre-update network state.
4. `0f107d1` / `e0bfdfd`: simultaneous hidden-layer perturbations reduce calibration from
   quadratic to linear depth work; batching directions makes the reduction visible in GPU wall
   time. `66e3034` fixes diagnostics so raw-apical ablations report the signal actually used.
5. `2f3f44d`: the raw-apical control can be matched per sample to the innovation norm. This
   separates subtraction/direction from a trivial update-magnitude explanation.
6. `6093fe5` / `d6a00aa`: FA, PEPITA, Forward-Forward, and EP were reimplemented against their
   papers and reference code. EP uses persistent free particles; deep PEPITA requires an explicit
   learning rate; FA now logs the signal actually transported through its fixed feedback path.

All older `deepres_*` files predate item 1. Their alignment directions remain informative, but
their accuracy and stability should not be pooled with the corrected runs below.

## Method
SDIL (Somato-Dendritic Innovation Learning), a local non-backprop rule inspired by Harnett 2026.
Per hidden layer l:
- forward: `u_l = W_l h_{l-1}`, `h_l = φ(u_l)`
- apical feedback `a_l = A_l c` (c = output error e = softmax−onehot, broadcast)
- default per-neuron predictor `ĥ_l = p_l ⊙ h_l + b_l`; teaching signal `r_l = a_l − ĥ_l`
- three-factor update `ΔW_l = η (r_l ⊙ φ'(u_l)) h_{l-1}^T`
- `A_l` learned by node perturbation (q_l ≈ −∇_{h_l}L, forward-only, no weight transport),
  following Lansdell, Prakash & Kording's learned synthetic-feedback method
- `P_l` learned on neutral (c=0) periods only, so it strips the soma-predictable nuisance
  without eating the teaching signal.

The scalable perturbation mode injects independent Rademacher interventions at all hidden layers
in the same antithetic forward evaluations. Cross-layer interference adds variance but averages to
zero; it trains all `A_l` using `O(depth)` rather than `O(depth^2)` forward work.

Relative to fixed DFA/FA, the feedback pathway is learned by causal perturbation. That ingredient
is not novel: without traffic and `P`, it is the direct learned-feedback method of
[Lansdell et al.](https://arxiv.org/abs/1906.00889). The candidate SDIL contribution is specifically
the per-cell residual under mixed apical traffic and its neutral-period identification. The full
claim boundary is frozen in `NOVELTY.md`.

## Critical fix
Node-perturbation estimator must divide by σ, not σ² (with δ=σξ, ĝ = δ·ΔL/σ² = ξ·ΔL/σ ⇒ ‖q‖≈‖∇L‖).
The σ² bug made the teaching signal ~100× too large ⇒ effective LR ~100× ⇒ SDIL stalled. Cosine is
magnitude-invariant so alignment looked fine while optimization blew up. Fixing it turned SDIL from
"worse than DFA everywhere" to "matches DFA on MNIST, beats it on FMNIST/CIFAR". (Codex independently
flagged the same σ-scale issue.)

## Verified five-depth scaling (CIFAR-10, width 64, 5 epochs, five seeds)

Mean ± sample SD. Alignment is the mean per-sample cosine over the earliest third of hidden layers
at the final checkpoint. SDIL uses simultaneous perturbations with 16 batched directions.

| depth | method | test acc (%) | early-third alignment | wall time (s) |
|------:|:-------|-------------:|----------------------:|--------------:|
| 5 | BP | 41.546 ± 0.422 | — | 6.9 ± 0.3 |
| 5 | FA | 38.924 ± 0.729 | 0.630 ± 0.022 | 5.7 ± 1.0 |
| 5 | DFA | 37.768 ± 0.654 | 0.514 ± 0.032 | 5.0 ± 0.2 |
| 5 | SDIL | 41.460 ± 0.226 | 0.797 ± 0.018 | 9.8 ± 1.9 |
| 10 | BP | 41.702 ± 0.730 | — | 11.7 ± 0.3 |
| 10 | FA | 39.056 ± 0.557 | 0.610 ± 0.022 | 10.0 ± 2.1 |
| 10 | DFA | 36.716 ± 0.363 | 0.231 ± 0.030 | 9.4 ± 1.7 |
| 10 | SDIL | 41.046 ± 0.222 | 0.496 ± 0.024 | 19.6 ± 4.4 |
| 20 | BP | 41.910 ± 0.560 | — | 20.2 ± 1.1 |
| 20 | FA | 37.448 ± 0.603 | 0.569 ± 0.025 | 18.8 ± 3.6 |
| 20 | DFA | 37.106 ± 0.710 | 0.126 ± 0.023 | 17.9 ± 3.5 |
| 20 | SDIL | 41.288 ± 0.606 | 0.403 ± 0.020 | 31.2 ± 1.5 |
| 30 | BP | 41.908 ± 0.697 | — | 29.8 ± 1.3 |
| 30 | FA | 38.158 ± 0.517 | 0.593 ± 0.014 | 26.9 ± 1.9 |
| 30 | DFA | 36.988 ± 0.524 | 0.087 ± 0.007 | 23.4 ± 0.8 |
| 30 | SDIL | 41.198 ± 0.523 | 0.366 ± 0.007 | 47.8 ± 1.4 |
| 60 | BP | 41.624 ± 0.781 | — | 58.4 ± 1.0 |
| 60 | FA | 38.428 ± 0.387 | 0.577 ± 0.011 | 49.8 ± 1.8 |
| 60 | DFA | 36.898 ± 0.444 | 0.047 ± 0.005 | 47.6 ± 5.6 |
| 60 | SDIL | 41.246 ± 0.323 | 0.336 ± 0.013 | 96.4 ± 12.4 |

Across paired seeds, SDIL changes by `-0.214 ± 0.349` accuracy points from d5 to d60, while the
depth grows 12x and wall time grows 9.87x (geometric mean). Its paired gap to BP is
`0.086 ± 0.346` points at d5 and `0.378 ± 0.927` at d60. The SDIL−DFA advantage stays between
3.69 and 4.35 points at every tested depth. Flattened CIFAR itself is depth-flat, so this establishes
preservation under depth, not a claim that extra depth improves the task.

The mean-based nondominated frontier among local methods is DFA-d5 (`5.01 s`, `37.77%`), FA-d5
(`5.72 s`, `38.92%`), then SDIL-d5 (`9.84 s`, `41.46%`). BP is shown only as a nonlocal reference.
Thus SDIL occupies the high-accuracy end of the audited local-learning frontier; the deeper SDIL
configurations demonstrate stability, but are correctly dominated by its d5 configuration on this
depth-flat task.

SDIL's early-third alignment declines from `0.797` at d5 to `0.336` at d60; it is useful but not
depth-invariant. DFA collapses much further (`0.514` to `0.047`). Residual FA does **not** show that
failure because identity skips transport most of the state gradient exactly (`0.630` to `0.577`),
yet its accuracy stays roughly 2.6–4.5 points below BP. The supported claim is therefore not “all FA
signals lose alignment”: direction, scale, serial transport, and optimization quality must be
separated.

## Harnett-specific residualization (MNIST, d3/w256, 15 epochs, five seeds)

The matched-raw control preserves the raw apical direction but rescales every sample to the
innovation's norm.

| nuisance rho | raw apical acc (%) | matched raw acc (%) | residual acc (%) |
|-------------:|-------------------:|--------------------:|-----------------:|
| 0.00 | 97.450 ± 0.046 | 97.450 ± 0.046 | 97.450 ± 0.046 |
| 0.05 | 96.438 ± 0.077 | 96.980 ± 0.148 | 97.382 ± 0.154 |
| 0.20 | 95.512 ± 0.150 | 95.974 ± 0.144 | 97.370 ± 0.083 |
| 0.50 | 10.380 ± 0.548 | 10.310 ± 0.633 | 97.348 ± 0.133 |

At zero nuisance all three signals are identical. As normal soma-dendrite coupling grows, raw
feedback degrades and eventually reaches chance; residual learning stays at 97.35–97.45%. Matching
the raw signal's norm recovers only part of the low-rho loss and does nothing at rho=0.5. The strong
result therefore depends on subtracting the soma-predictable component, not merely clipping or
normalizing an oversized update. This is the most direct evidence that the Harnett-specific
innovation operation is algorithmically necessary.

## Legacy accuracy (pre-audit implementation)
| task                         | BP        | DFA   | SDIL          |
|------------------------------|-----------|-------|---------------|
| MNIST  d3 / d5 / d7 / d10    | 98.3      | 97.3 / 97.2 / 97.1 / 96.8 | 97.4 / 97.3 / 97.1 / 97.1 |
| FashionMNIST d3 / d5 / d7 / d10 | 88–89  | 86.3 / 87.1 / 86.2 / 86.9 | 88.4 / 87.6 / 87.3 / 88.1 |
| CIFAR-10 (no-BN residual) d5/10/20 | 44.2 / 44.8 / 45.5 | 40.6 / 39.9 / 37.4 | 42.4 / 42.6 / 42.4 |

SDIL matches DFA on easy MNIST and **beats DFA on FashionMNIST (+1–2 pts) and CIFAR (+2 to +5 pts)**.
On CIFAR the SDIL−DFA gap GROWS with depth (+1.8 → +2.7 → +5.0 at d5/10/20): DFA degrades with depth,
SDIL holds.

## Endogenous top-down traffic timescale screen (validation only)

This development screen used one model seed, a fixed 5,000-example stratified MNIST validation
split drawn only from training data, and never evaluated the test set. All 18 runs share validation
index hash `c4cec216...2e3d`, clean source revision `17bcd97`, and the same 15-epoch training
protocol. The grid crossed `eta_P in {0.002, 0.01, 0.05}`, neutral warmup steps in
`{0, 200, 1000}`, and endogenous top-down traffic scales in `{0.2, 0.5}`.

| eta_P | warmup steps | rho=0.2 last val (%) | rho=0.5 last val (%) | mean (%) | worst (%) |
|------:|-------------:|---------------------:|---------------------:|---------:|----------:|
| 0.002 | 0 | 91.50 | 77.24 | 84.37 | 77.24 |
| 0.002 | 200 | 91.02 | 89.10 | 90.06 | 89.10 |
| 0.002 | 1000 | 93.10 | 88.82 | 90.96 | 88.82 |
| 0.010 | 0 | 94.94 | 92.40 | 93.67 | 92.40 |
| 0.010 | 200 | 92.88 | 91.12 | 92.00 | 91.12 |
| 0.010 | 1000 | 95.04 | 91.06 | 93.05 | 91.06 |
| 0.050 | 0 | 96.52 | 91.40 | 93.96 | 91.40 |
| **0.050** | **200** | **94.20** | **95.26** | **94.73** | **94.20** |
| 0.050 | 1000 | 94.70 | 93.62 | 94.16 | 93.62 |

The selected development setting is `eta_P=0.05`, 200 warmup steps: it has the highest mean
last-epoch validation accuracy and the highest worst-traffic endpoint. Its final mean traffic R2 is
0.802 at rho=0.2 and 0.999 at rho=0.5. This reverses the initial assumption that `P` must be much
slower than forward learning: once teaching and traffic are separated by neutral periods, fast
tracking is beneficial, while a long warmup is not consistently useful. This is a frozen
validation choice for the multi-seed C1 experiment, not evidence that C1 has passed.

## Causal-calibration query screen (validation only)

This seed-0 development screen used CIFAR-10 d20/w64 for five epochs and a 5,000-example
training-only validation split (hash `8328b206...515b`). All eight runs came from clean revision
`1a22a99`; the test set was never evaluated. Gain retention is
`(candidate - DFA) / (K16/e4 - DFA)` at the final validation endpoint.

| method | directions K | every | val (%) | batch loss queries | query reduction | calibration work reduction | gain retention |
|:-------|-------------:|------:|--------:|-------------------:|----------------:|---------------------------:|---------------:|
| DFA | — | — | 36.66 | 0 | — | — | — |
| learned NP | 16 | 4 | 37.46 | 14,080 | 1.0x | 1.0x | 1.000 |
| **learned NP** | **1** | **4** | **38.08** | **880** | **16.0x** | **11.0x** | **1.775** |
| learned NP | 2 | 8 | 37.32 | 880 | 16.0x | 13.2x | 0.825 |
| learned NP | 4 | 16 | 35.98 | 880 | 16.0x | 14.7x | -0.850 |
| learned NP | 1 | 8 | 36.70 | 440 | 32.0x | 22.0x | 0.050 |
| learned NP | 1 | 16 | 36.52 | 220 | 64.0x | 44.0x | -0.175 |
| learned NP | 2 | 32 | 36.78 | 220 | 64.0x | 52.8x | 0.150 |

`K=1/every=4` is frozen for confirmation: it exceeds the C3 10x-query/90%-retention threshold in
validation and is the only candidate that improves on the reference endpoint. The ablation also
separates directions from update frequency: spending the same 880 queries as `K=2/every=8` or
`K=4/every=16` is worse, so frequent noisy regression targets are more useful than precise sparse
targets. GTX-1080 wall time remains approximately 15–23 seconds across the grid despite the 11x
calibration-work change, demonstrating why the Pareto audit needs hardware-independent work in
addition to wall time. C3 is not declared passed until frozen multi-seed confirmation.

## Legacy credit assignment / depth utility
Per-layer alignment cos(r_l, −∇_{h_l}L), early-third average, CIFAR no-BN residual:
| depth | DFA early-align | SDIL early-align |
|-------|-----------------|------------------|
| 5     | 0.55            | 0.83             |
| 10    | 0.36            | 0.86             |
| 20    | 0.25            | 0.88             |
| 30    | 0.19            | 0.87             |

DFA's teaching signal to early/deep layers collapses toward random (~0.05–0.19) as depth grows;
SDIL stays ~0.85 and is depth-invariant. This is the "does credit reach early layers" metric that
DFA fails and SDIL passes — the core advantage.

## Paper-faithful non-backprop baselines

`PETITE` in the initial request is treated as **PEPITA**. Implementations and hyperparameters were
checked against the [FA paper](https://www.nature.com/articles/ncomms13276),
[PEPITA paper](https://proceedings.mlr.press/v162/dellaferrera22a.html) and
[author code](https://github.com/GiorgiaD/PEPITA),
[Forward-Forward paper](https://www.cs.toronto.edu/~hinton/absps/FFXfinal.pdf) and
[reference implementation](https://github.com/mpezeshki/pytorch_forward_forward), and the
[EP paper](https://www.frontiersin.org/journals/computational-neuroscience/articles/10.3389/fncom.2017.00024/full)
and [author code](https://github.com/bscellier/Towards-a-Biologically-Plausible-Backprop).

Native protocol results are not an equal-compute comparison:

| method | architecture | budget | n | last test acc (%) | wall time (s) |
|:-------|:-------------|:-------|--:|------------------:|--------------:|
| PEPITA | d3/w256 | 60 epochs | 5 | 91.094 ± 0.214 | 89.2 ± 3.5 |
| Forward-Forward | d3/w256 | 15 epochs/layer | 5 | 93.558 ± 0.232 | 68.2 ± 0.8 |
| EP | d1/w500 | 25 epochs | 5 | 96.926 ± 0.238 | 800.5 ± 53.3 |
| EP | d2/w500 | 60 epochs | 5 | 89.526 ± 4.461 | 17530.4 ± 613.6 |

PEPITA uses its two forward presentations, He-uniform initialization, shared 10% dropout mask, and
an explicit deep-network learning rate of 0.001. Forward-Forward trains greedy local goodness layers
and has no classifier readout. EP uses the original hard-sigmoid energy dynamics, random beta sign,
and persistent training particles on the fixed first 50k examples. Its published d1 schedule uses
20 free steps, 4 nudged steps, `beta=0.5`, and layer rates `[0.1, 0.05]`; d2 uses 150 free steps,
6 nudged steps, `beta=1`, and rates `[0.4, 0.1, 0.01]`. Evaluation starts fresh zero particles and
runs the full free relaxation. The shallow EP protocol is the strongest native baseline here. The
canonical deep protocol is much more expensive and highly initialization-sensitive in these runs.

### EP exact-architecture comparison

All methods below use the same first 50k MNIST examples and the same epoch budget within each exact
forward architecture: 25 epochs for d1/w500 (397,510 parameters) and 60 for d2/w500 (648,010
parameters). EP retains its native raw-pixel hard-sigmoid energy protocol; BP/DFA/SDIL retain the
shared z-score/tanh feedforward protocol.

| depth | method | n | last test acc (%) | best test acc (%) | wall time (s) |
|:------|:-------|--:|------------------:|------------------:|--------------:|
| d1 | BP | 5 | 98.240 ± 0.073 | 98.272 ± 0.080 | 15.0 ± 0.5 |
| d1 | DFA | 5 | 97.662 ± 0.078 | 97.700 ± 0.074 | 10.6 ± 1.6 |
| d1 | EP | 5 | 96.926 ± 0.238 | 97.198 ± 0.147 | 800.5 ± 53.3 |
| d1 | SDIL | 5 | 97.596 ± 0.089 | 97.638 ± 0.075 | 15.5 ± 0.2 |
| d2 | BP | 5 | 98.288 ± 0.030 | 98.326 ± 0.036 | 45.1 ± 0.8 |
| d2 | DFA | 5 | 97.590 ± 0.046 | 97.622 ± 0.036 | 34.6 ± 2.8 |
| d2 | EP | 5 | 89.526 ± 4.461 | 90.028 ± 4.272 | 17530.4 ± 613.6 |
| d2 | SDIL | 5 | 97.516 ± 0.067 | 97.562 ± 0.058 | 58.5 ± 4.0 |

The paired SDIL−EP last-epoch gap is `0.670 ± 0.253` points at d1 and `7.990 ± 4.459` at d2;
SDIL's geometric-mean wall-time speedup is 51.7x and 299.8x, respectively. The five d2 EP outcomes
are 97.46%, 87.56%, 88.07%, 87.77%, and 86.77%. They are finite completed trajectories, not crashes:
one seed reaches the shallow-level basin while four do not. This is evidence of initialization
sensitivity under the reproduced canonical schedule, not evidence that EP is intrinsically
unstable across implementations. Reproducing the basin split in the original Theano revision or a
strong modern EP variant remains necessary. DFA is also strong on easy MNIST, so the scaling claim
comes from the five-depth CIFAR panel; this comparison establishes that SDIL need not trade away
accuracy to obtain a large cost advantage over canonical EP.

## How to run
`experiments/run.py --mode {bp,fa,dfa,sdil} --dataset {mnist,fmnist,cifar10} --depth D --residual {0,1} --act {tanh,gelu,silu,relu}`
Batteries: `experiments/run_v2.sh <ds> "<depths>" <res> <act> "<seeds>" <ep> <pfx>`.
Five-depth scaling: `experiments/figure_scaling_sweep.sh`; canonical local baselines:
`experiments/baseline_sweep.sh`; original d1 EP:
`experiments/ep_original_sweep.sh`; exact d1 comparison: `experiments/ep_archmatch_sweep.sh`;
canonical d2 comparison: `experiments/ep_depth2_sweep.sh`; near-parameter comparison:
`experiments/ep_matched_sweep.sh`.
Audited aggregate tables: `experiments/analyze_verified.py`. The claim-locked finalizer is
`experiments/finalize_claims.sh`; it rejects missing/dirty inputs, regenerates both main figures,
their captions, source-hash manifest, and `results/audited_tables.md`, then runs all smoke checks.

## Open items

- Treat learned direct node-perturbation feedback (Lansdell et al.) as the exact backbone baseline,
  and BurstCCN as the strongest dendritic/scaling baseline; do not attribute their ingredients to
  SDIL.
- Compare simultaneous calibration at fixed loss-evaluation budgets and sweep directions
  (4/8/16/32).
- Validate SDIL on a genuinely depth-necessary compositional task and a CNN; flattened CIFAR does
  not gain accuracy with depth.
- Diagnose why residual FA has high directional cosine but lower accuracy: log update-norm ratios,
  parameter-space cosine, and serial feedback latency/cost.
- Reproduce the d2 EP basin sensitivity in the original Theano revision and/or a strong modern EP
  variant without selectively removing seeds.
- Tune SDIL schedules using a held-out validation split, never the test trajectory, then freeze the
  protocol before expanding the EP comparison.
- Extend PEPITA and Forward-Forward beyond shallow MNIST before making any scaling claim about them.