# SDIL — results log (seed 0 unless noted; runs on timan107 GTX-1080 / ep_pascal)

## 2026-07-21 audit and version boundary

Git was initialized after inheriting the project. The original code/results are preserved at
`215acff`. Four audit findings changed the canonical implementation:

1. `49b86d6`: residual-block local updates now include the branch Jacobian
   `res_alpha=1/sqrt(depth)`. The omission preserved cosine direction but inflated update norms by
   `sqrt(depth)`, confounding cross-depth optimization comparisons.
2. `5255166`: the Harnett baseline is now a per-neuron affine soma-dendrite fit
   `ahat_l = p_l * h_l + b_l`, rather than a dense population map. This follows the paper's
   per-cell least-squares somatic-event to dendritic-event relation.
3. `c2769e6`: causal perturbation targets and predicted residuals are measured on the same
   pre-update network state.
4. `0f107d1` / `e0bfdfd`: simultaneous hidden-layer perturbations reduce calibration from
   quadratic to linear depth work; batching directions makes the reduction visible in GPU wall
   time. `66e3034` fixes diagnostics so raw-apical ablations report the signal actually used.

All older `deepres_*` files predate item 1. Their alignment directions remain informative, but
their accuracy and stability should not be pooled with the corrected runs below.

## Method
SDIL (Somato-Dendritic Innovation Learning), a local non-backprop rule inspired by Harnett 2026.
Per hidden layer l:
- forward: `u_l = W_l h_{l-1}`, `h_l = φ(u_l)`
- apical feedback `a_l = A_l c` (c = output error e = softmax−onehot, broadcast)
- default per-neuron predictor `ĥ_l = p_l ⊙ h_l + b_l`; teaching signal `r_l = a_l − ĥ_l`
- three-factor update `ΔW_l = η (r_l ⊙ φ'(u_l)) h_{l-1}^T`
- `A_l` LEARNED by amortized node perturbation (q_l ≈ −∇_{h_l}L, forward-only, no weight transport)
- `P_l` learned on neutral (c=0) periods only, so it strips the soma-predictable nuisance
  without eating the teaching signal.

The scalable perturbation mode injects independent Rademacher interventions at all hidden layers
in the same antithetic forward evaluations. Cross-layer interference adds variance but averages to
zero; it trains all `A_l` using `O(depth)` rather than `O(depth^2)` forward work.

Distinct from the failed `~/sdrn` (fixed-random feedback + post-hoc residualization) and from
Dual Prop / DFA / FA: the feedback pathway is *learned by causal perturbation*, so the residual is
aligned by construction rather than being a random projection.

## Critical fix
Node-perturbation estimator must divide by σ, not σ² (with δ=σξ, ĝ = δ·ΔL/σ² = ξ·ΔL/σ ⇒ ‖q‖≈‖∇L‖).
The σ² bug made the teaching signal ~100× too large ⇒ effective LR ~100× ⇒ SDIL stalled. Cosine is
magnitude-invariant so alignment looked fine while optimization blew up. Fixing it turned SDIL from
"worse than DFA everywhere" to "matches DFA on MNIST, beats it on FMNIST/CIFAR". (Codex independently
flagged the same σ-scale issue.)

## Corrected narrow depth pilot (CIFAR-10, width 64, 5 epochs, seed 0)

| depth | method | test acc | early-third alignment | wall time |
|------:|--------|---------:|----------------------:|----------:|
| 30 | BP | 41.31% | — | 29.4 s |
| 30 | DFA | 37.31% | 0.084 | 19.4 s |
| 30 | SDIL layerwise | 42.01% | 0.371 | 339.5 s |
| 30 | SDIL multiplexed+batched | 41.46% | 0.370 | 47.3 s |
| 60 | BP | 42.93% | — | 63.7 s |
| 60 | DFA | 37.31% | 0.041 | 41.1 s |
| 60 | SDIL layerwise | 40.79% | 0.328 | 1238.5 s |
| 60 | SDIL multiplexed+batched | 41.10% | 0.332 | 90.3 s |

Narrowing the model exposed the performance difference that width-256 flattened CIFAR had hidden.
DFA alignment halves from depth 30 to 60, while both SDIL estimators remain near 0.33–0.37. Batched
multiplexing preserves alignment while accelerating SDIL 7.2x at d30 and 13.7x at d60. Its wall
growth from d30 to d60 is 1.91x, versus 3.65x for layerwise calibration. These are pilot, single-seed
results; the ~0.5-point layerwise/multiplexed accuracy difference is not yet statistically resolved.

## Harnett-specific residualization (MNIST, 15 epochs, seed 0)

| nuisance rho | raw apical acc | residual acc | raw teaching alignment | residual alignment |
|-------------:|---------------:|-------------:|-----------------------:|-------------------:|
| 0.00 | 97.31% | 97.31% | 0.138 | 0.138 |
| 0.02 | 96.90% | 97.27% | 0.096 | 0.156 |
| 0.05 | 96.48% | 97.24% | 0.092 | 0.160 |
| 0.10 | 95.87% | 97.32% | 0.068 | 0.140 |
| 0.20 | 95.60% | 97.31% | 0.103 | 0.147 |
| 0.50 | 8.92% | 97.18% | 0.000 | 0.149 |
| 1.00 | 8.92% | 97.29% | 0.000 | 0.140 |
| 2.00 | 9.82% | 97.31% | 0.000 | 0.121 |

At zero nuisance, subtraction is a no-op. As normal soma-dendrite coupling grows, raw feedback
degrades continuously and eventually enters a self-reinforcing Hebbian failure, while the learned
innovation remains stable. The low-rho sweep is important: it shows a graded 0.37→1.71 point gap
before the strong-contamination phase transition, rather than only an artificial chance-level
contrast.

## Legacy accuracy (pre-audit implementation)
| task                         | BP        | DFA   | SDIL          |
|------------------------------|-----------|-------|---------------|
| MNIST  d3 / d5 / d7 / d10    | 98.3      | 97.3 / 97.2 / 97.1 / 96.8 | 97.4 / 97.3 / 97.1 / 97.1 |
| FashionMNIST d3 / d5 / d7 / d10 | 88–89  | 86.3 / 87.1 / 86.2 / 86.9 | 88.4 / 87.6 / 87.3 / 88.1 |
| CIFAR-10 (no-BN residual) d5/10/20 | 44.2 / 44.8 / 45.5 | 40.6 / 39.9 / 37.4 | 42.4 / 42.6 / 42.4 |

SDIL matches DFA on easy MNIST and **beats DFA on FashionMNIST (+1–2 pts) and CIFAR (+2 to +5 pts)**.
On CIFAR the SDIL−DFA gap GROWS with depth (+1.8 → +2.7 → +5.0 at d5/10/20): DFA degrades with depth,
SDIL holds.

## Legacy credit assignment / depth utility
Per-layer alignment cos(r_l, −∇_{h_l}L), early-third average, CIFAR no-BN residual:
| depth | DFA early-align | SDIL early-align |
|-------|-----------------|------------------|
| 5     | 0.55            | 0.83             |
| 10    | 0.36            | 0.86             |
| 20    | 0.25            | 0.88             |
| 30    | 0.19            | 0.87             |

DFA's teaching signal to early/deep layers collapses toward random (~0.05–0.19) as depth grows;
SDIL stays ~0.85 and is depth-invariant. This is the "does credit reach early layers" metric that
DFA fails and SDIL passes — the core advantage.

## Baselines (`sdil/local_baselines.py`)
- FA (Lillicrap, sequential random feedback): WORKS — 94.7% MNIST (1 epoch). Included.
- DFA: SDIL with fixed-random A. Included (primary comparison).
- EP (Equilibrium Propagation, Scellier & Bengio 2017): WORKS — 87% MNIST on an 8k subset with
  β=0.5, feedforward warm-start settling; ~95%+ expected on full data. Included.
- PEPITA (forward-only error modulation): implemented, undertuned (~19%). Needs work.
- Forward-Forward (Hinton): implemented; per-layer goodness separates but inference is off. Needs work.

## How to run
`experiments/run.py --mode {bp,dfa,sdil} --dataset {mnist,fmnist,cifar10} --depth D --residual {0,1} --act {tanh,gelu,silu,relu}`
Batteries: `experiments/run_v2.sh <ds> "<depths>" <res> <act> "<seeds>" <ep> <pfx>`.
Analysis: `experiments/analyze_depth.py {depth,deepres}` → tables + figures in `results/figs/`.
Smoke/mechanism checks: `experiments/smoke.py`.

## Open items
- Repeat corrected width-64 d30/d60 and nuisance transition runs for seeds 1–4.
- Compare multiplexed calibration at fixed loss-evaluation budgets and sweep directions (4/8/16/32).
- Add a magnitude-matched or clipped raw-apical control to separate directional contamination from
  simple update explosion at large nuisance.
- Validate simultaneous perturbation on a genuinely depth-necessary compositional task and a CNN,
  not only flattened CIFAR.
- FA's residual-network implementation and the current EP prototype are not yet strong enough for a
  publication-quality scaling comparison; do not overstate those baselines.