# SDIL — results log (seed 0 unless noted; runs on timan107 GTX-1080 / ep_pascal)

## Method
SDIL (Somato-Dendritic Innovation Learning), a local non-backprop rule inspired by Harnett 2026.
Per hidden layer l:
- forward: `u_l = W_l h_{l-1}`, `h_l = φ(u_l)`
- apical feedback `a_l = A_l c` (c = output error e = softmax−onehot, broadcast)
- predictor `ĥ_l = P_l h_l`; teaching signal (innovation) `r_l = a_l − ĥ_l`
- three-factor update `ΔW_l = η (r_l ⊙ φ'(u_l)) h_{l-1}^T`
- `A_l` LEARNED by amortized node perturbation (q_l ≈ −∇_{h_l}L, forward-only, no weight transport)
- `P_l` learned on neutral (c=0) periods only, so it strips the soma-predictable nuisance
  without eating the teaching signal.

Distinct from the failed `~/sdrn` (fixed-random feedback + post-hoc residualization) and from
Dual Prop / DFA / FA: the feedback pathway is *learned by causal perturbation*, so the residual is
aligned by construction rather than being a random projection.

## Critical fix
Node-perturbation estimator must divide by σ, not σ² (with δ=σξ, ĝ = δ·ΔL/σ² = ξ·ΔL/σ ⇒ ‖q‖≈‖∇L‖).
The σ² bug made the teaching signal ~100× too large ⇒ effective LR ~100× ⇒ SDIL stalled. Cosine is
magnitude-invariant so alignment looked fine while optimization blew up. Fixing it turned SDIL from
"worse than DFA everywhere" to "matches DFA on MNIST, beats it on FMNIST/CIFAR". (Codex independently
flagged the same σ-scale issue.)

## Accuracy (test %, seed 0)
| task                         | BP        | DFA   | SDIL          |
|------------------------------|-----------|-------|---------------|
| MNIST  d3 / d5 / d7 / d10    | 98.3      | 97.3 / 97.2 / 97.1 / 96.8 | 97.4 / 97.3 / 97.1 / 97.1 |
| FashionMNIST d3 / d5 / d7 / d10 | 88–89  | 86.3 / 87.1 / 86.2 / 86.9 | 88.4 / 87.6 / 87.3 / 88.1 |
| CIFAR-10 (no-BN residual) d5/10/20 | 44.2 / 44.8 / 45.5 | 40.6 / 39.9 / 37.4 | 42.4 / 42.6 / 42.4 |

SDIL matches DFA on easy MNIST and **beats DFA on FashionMNIST (+1–2 pts) and CIFAR (+2 to +5 pts)**.
On CIFAR the SDIL−DFA gap GROWS with depth (+1.8 → +2.7 → +5.0 at d5/10/20): DFA degrades with depth,
SDIL holds.

## Credit assignment / depth utility — the mechanistic result
Per-layer alignment cos(r_l, −∇_{h_l}L), early-third average, CIFAR no-BN residual:
| depth | DFA early-align | SDIL early-align |
|-------|-----------------|------------------|
| 5     | 0.55            | 0.83             |
| 10    | 0.36            | 0.86             |
| 20    | 0.25            | 0.88             |
| 30    | 0.19            | (running)        |

DFA's teaching signal to early/deep layers collapses toward random (~0.05–0.19) as depth grows;
SDIL stays ~0.85 and is depth-invariant. This is the "does credit reach early layers" metric that
DFA fails and SDIL passes — the core advantage.

## Baselines (`sdil/local_baselines.py`)
- FA (Lillicrap, sequential random feedback): WORKS — 94.7% MNIST (1 epoch). Included.
- DFA: SDIL with fixed-random A. Included (primary comparison).
- EP (Equilibrium Propagation, Scellier & Bengio 2017): WORKS — 87% MNIST on an 8k subset with
  β=0.5, feedforward warm-start settling; ~95%+ expected on full data. Included.
- PEPITA (forward-only error modulation): implemented, undertuned (~19%). Needs work.
- Forward-Forward (Hinton): implemented; per-layer goodness separates but inference is off. Needs work.

## How to run
`experiments/run.py --mode {bp,dfa,sdil} --dataset {mnist,fmnist,cifar10} --depth D --residual {0,1} --act {tanh,gelu,silu,relu}`
Batteries: `experiments/run_v2.sh <ds> "<depths>" <res> <act> "<seeds>" <ep> <pfx>`.
Analysis: `experiments/analyze_depth.py {depth,deepres}` → tables + figures in `results/figs/`.
Smoke/mechanism checks: `experiments/smoke.py`.

## Open items
- Complete CIFAR SDIL d30 (running) + seeds 1,2 (running) for mean±std.
- Codex-suggested boosts (not yet applied): separate normalized feedback direction from a learned
  per-layer scalar gain; Adam-like second-moment on local updates; forward-only gradient-norm gain
  calibration. Current results already beat DFA without these.
- GRAPE (ICLR2026) read for DFA-failure regimes only; treated as unverified, not a baseline.