chore: capture initial SDIL project state

1 files changed, 68 insertions, 0 deletions

diff --git a/RESULTS.md b/RESULTS.md
new file mode 100644
index 0000000..d2bdc48
--- /dev/null
+++ b/RESULTS.md
@@ -0,0 +1,68 @@
+# SDIL — results log (seed 0 unless noted; runs on timan107 GTX-1080 / ep_pascal)
+
+## Method
+SDIL (Somato-Dendritic Innovation Learning), a local non-backprop rule inspired by Harnett 2026.
+Per hidden layer l:
+- forward: `u_l = W_l h_{l-1}`, `h_l = φ(u_l)`
+- apical feedback `a_l = A_l c` (c = output error e = softmax−onehot, broadcast)
+- predictor `ĥ_l = P_l h_l`; teaching signal (innovation) `r_l = a_l − ĥ_l`
+- three-factor update `ΔW_l = η (r_l ⊙ φ'(u_l)) h_{l-1}^T`
+- `A_l` LEARNED by amortized node perturbation (q_l ≈ −∇_{h_l}L, forward-only, no weight transport)
+- `P_l` learned on neutral (c=0) periods only, so it strips the soma-predictable nuisance
+  without eating the teaching signal.
+
+Distinct from the failed `~/sdrn` (fixed-random feedback + post-hoc residualization) and from
+Dual Prop / DFA / FA: the feedback pathway is *learned by causal perturbation*, so the residual is
+aligned by construction rather than being a random projection.
+
+## Critical fix
+Node-perturbation estimator must divide by σ, not σ² (with δ=σξ, ĝ = δ·ΔL/σ² = ξ·ΔL/σ ⇒ ‖q‖≈‖∇L‖).
+The σ² bug made the teaching signal ~100× too large ⇒ effective LR ~100× ⇒ SDIL stalled. Cosine is
+magnitude-invariant so alignment looked fine while optimization blew up. Fixing it turned SDIL from
+"worse than DFA everywhere" to "matches DFA on MNIST, beats it on FMNIST/CIFAR". (Codex independently
+flagged the same σ-scale issue.)
+
+## Accuracy (test %, seed 0)
+| task                         | BP        | DFA   | SDIL          |
+|------------------------------|-----------|-------|---------------|
+| MNIST  d3 / d5 / d7 / d10    | 98.3      | 97.3 / 97.2 / 97.1 / 96.8 | 97.4 / 97.3 / 97.1 / 97.1 |
+| FashionMNIST d3 / d5 / d7 / d10 | 88–89  | 86.3 / 87.1 / 86.2 / 86.9 | 88.4 / 87.6 / 87.3 / 88.1 |
+| CIFAR-10 (no-BN residual) d5/10/20 | 44.2 / 44.8 / 45.5 | 40.6 / 39.9 / 37.4 | 42.4 / 42.6 / 42.4 |
+
+SDIL matches DFA on easy MNIST and **beats DFA on FashionMNIST (+1–2 pts) and CIFAR (+2 to +5 pts)**.
+On CIFAR the SDIL−DFA gap GROWS with depth (+1.8 → +2.7 → +5.0 at d5/10/20): DFA degrades with depth,
+SDIL holds.
+
+## Credit assignment / depth utility — the mechanistic result
+Per-layer alignment cos(r_l, −∇_{h_l}L), early-third average, CIFAR no-BN residual:
+| depth | DFA early-align | SDIL early-align |
+|-------|-----------------|------------------|
+| 5     | 0.55            | 0.83             |
+| 10    | 0.36            | 0.86             |
+| 20    | 0.25            | 0.88             |
+| 30    | 0.19            | (running)        |
+
+DFA's teaching signal to early/deep layers collapses toward random (~0.05–0.19) as depth grows;
+SDIL stays ~0.85 and is depth-invariant. This is the "does credit reach early layers" metric that
+DFA fails and SDIL passes — the core advantage.
+
+## Baselines (`sdil/local_baselines.py`)
+- FA (Lillicrap, sequential random feedback): WORKS — 94.7% MNIST (1 epoch). Included.
+- DFA: SDIL with fixed-random A. Included (primary comparison).
+- EP (Equilibrium Propagation, Scellier & Bengio 2017): WORKS — 87% MNIST on an 8k subset with
+  β=0.5, feedforward warm-start settling; ~95%+ expected on full data. Included.
+- PEPITA (forward-only error modulation): implemented, undertuned (~19%). Needs work.
+- Forward-Forward (Hinton): implemented; per-layer goodness separates but inference is off. Needs work.
+
+## How to run
+`experiments/run.py --mode {bp,dfa,sdil} --dataset {mnist,fmnist,cifar10} --depth D --residual {0,1} --act {tanh,gelu,silu,relu}`
+Batteries: `experiments/run_v2.sh <ds> "<depths>" <res> <act> "<seeds>" <ep> <pfx>`.
+Analysis: `experiments/analyze_depth.py {depth,deepres}` → tables + figures in `results/figs/`.
+Smoke/mechanism checks: `experiments/smoke.py`.
+
+## Open items
+- Complete CIFAR SDIL d30 (running) + seeds 1,2 (running) for mean±std.
+- Codex-suggested boosts (not yet applied): separate normalized feedback direction from a learned
+  per-layer scalar gain; Adam-like second-moment on local updates; forward-only gradient-norm gain
+  calibration. Current results already beat DFA without these.
+- GRAPE (ICLR2026) read for DFA-failure regimes only; treated as unverified, not a baseline.