docs: report final audited scaling and EP results

1 files changed, 71 insertions, 38 deletions

diff --git a/RESULTS.md b/RESULTS.md
index 4e91cc8..cdbe4d7 100644
--- a/RESULTS.md
+++ b/RESULTS.md
@@ -51,13 +51,25 @@ magnitude-invariant so alignment looked fine while optimization blew up. Fixing
 "worse than DFA everywhere" to "matches DFA on MNIST, beats it on FMNIST/CIFAR". (Codex independently
 flagged the same σ-scale issue.)
 
-## Verified narrow depth scaling (CIFAR-10, width 64, 5 epochs, five seeds)
+## Verified five-depth scaling (CIFAR-10, width 64, 5 epochs, five seeds)
 
 Mean ± sample SD. Alignment is the mean per-sample cosine over the earliest third of hidden layers
 at the final checkpoint. SDIL uses simultaneous perturbations with 16 batched directions.
 
 | depth | method | test acc (%) | early-third alignment | wall time (s) |
 |------:|:-------|-------------:|----------------------:|--------------:|
+| 5 | BP | 41.546 ± 0.422 | — | 6.9 ± 0.3 |
+| 5 | FA | 38.924 ± 0.729 | 0.630 ± 0.022 | 5.7 ± 1.0 |
+| 5 | DFA | 37.768 ± 0.654 | 0.514 ± 0.032 | 5.0 ± 0.2 |
+| 5 | SDIL | 41.460 ± 0.226 | 0.797 ± 0.018 | 9.8 ± 1.9 |
+| 10 | BP | 41.702 ± 0.730 | — | 11.7 ± 0.3 |
+| 10 | FA | 39.056 ± 0.557 | 0.610 ± 0.022 | 10.0 ± 2.1 |
+| 10 | DFA | 36.716 ± 0.363 | 0.231 ± 0.030 | 9.4 ± 1.7 |
+| 10 | SDIL | 41.046 ± 0.222 | 0.496 ± 0.024 | 19.6 ± 4.4 |
+| 20 | BP | 41.910 ± 0.560 | — | 20.2 ± 1.1 |
+| 20 | FA | 37.448 ± 0.603 | 0.569 ± 0.025 | 18.8 ± 3.6 |
+| 20 | DFA | 37.106 ± 0.710 | 0.126 ± 0.023 | 17.9 ± 3.5 |
+| 20 | SDIL | 41.288 ± 0.606 | 0.403 ± 0.020 | 31.2 ± 1.5 |
 | 30 | BP | 41.908 ± 0.697 | — | 29.8 ± 1.3 |
 | 30 | FA | 38.158 ± 0.517 | 0.593 ± 0.014 | 26.9 ± 1.9 |
 | 30 | DFA | 36.988 ± 0.524 | 0.087 ± 0.007 | 23.4 ± 0.8 |
@@ -67,17 +79,24 @@ at the final checkpoint. SDIL uses simultaneous perturbations with 16 batched di
 | 60 | DFA | 36.898 ± 0.444 | 0.047 ± 0.005 | 47.6 ± 5.6 |
 | 60 | SDIL | 41.246 ± 0.323 | 0.336 ± 0.013 | 96.4 ± 12.4 |
 
-The paired SDIL−DFA accuracy gap is `4.210 ± 0.426` points at d30 and
-`4.348 ± 0.554` at d60. SDIL wall time grows 2.02x when depth doubles, so the batched simultaneous
-calibration removes the former superlinear depth cost. Its accuracy is depth-stable and remains
-within 0.71 points of BP at d30 and 0.38 points at d60. Flattened CIFAR itself is depth-flat here, so
-these runs establish preservation rather than positive accuracy scaling.
-
-DFA's early-layer direction nearly halves with depth. Residual FA does **not** show that failure:
-its identity skips transport most of the state gradient exactly, leaving cosine near 0.58–0.59.
-Nevertheless FA plateaus around 38.3%, roughly three points below SDIL. Thus the supported claim is
-not “all FA signals lose directional alignment”; direction, signal scale, serial transport, and
-optimization efficiency must be separated.
+Across paired seeds, SDIL changes by `-0.214 ± 0.349` accuracy points from d5 to d60, while the
+depth grows 12x and wall time grows 9.87x (geometric mean). Its paired gap to BP is
+`0.086 ± 0.346` points at d5 and `0.378 ± 0.927` at d60. The SDIL−DFA advantage stays between
+3.69 and 4.35 points at every tested depth. Flattened CIFAR itself is depth-flat, so this establishes
+preservation under depth, not a claim that extra depth improves the task.
+
+The mean-based nondominated frontier among local methods is DFA-d5 (`5.01 s`, `37.77%`), FA-d5
+(`5.72 s`, `38.92%`), then SDIL-d5 (`9.84 s`, `41.46%`). BP is shown only as a nonlocal reference.
+Thus SDIL occupies the high-accuracy end of the audited local-learning frontier; the deeper SDIL
+configurations demonstrate stability, but are correctly dominated by its d5 configuration on this
+depth-flat task.
+
+SDIL's early-third alignment declines from `0.797` at d5 to `0.336` at d60; it is useful but not
+depth-invariant. DFA collapses much further (`0.514` to `0.047`). Residual FA does **not** show that
+failure because identity skips transport most of the state gradient exactly (`0.630` to `0.577`),
+yet its accuracy stays roughly 2.6–4.5 points below BP. The supported claim is therefore not “all FA
+signals lose alignment”: direction, scale, serial transport, and optimization quality must be
+separated.
 
 ## Harnett-specific residualization (MNIST, d3/w256, 15 epochs, five seeds)
 
@@ -139,42 +158,57 @@ Native protocol results are not an equal-compute comparison:
 |:-------|:-------------|:-------|--:|------------------:|--------------:|
 | PEPITA | d3/w256 | 60 epochs | 5 | 91.094 ± 0.214 | 89.2 ± 3.5 |
 | Forward-Forward | d3/w256 | 15 epochs/layer | 5 | 93.558 ± 0.232 | 68.2 ± 0.8 |
-| EP | d1/w500 | 25 epochs | 2 | 97.040 ± 0.269 | 839.8 ± 78.4 |
+| EP | d1/w500 | 25 epochs | 5 | 96.926 ± 0.238 | 800.5 ± 53.3 |
+| EP | d2/w500 | 60 epochs | 5 | 89.526 ± 4.461 | 17530.4 ± 613.6 |
 
 PEPITA uses its two forward presentations, He-uniform initialization, shared 10% dropout mask, and
 an explicit deep-network learning rate of 0.001. Forward-Forward trains greedy local goodness layers
 and has no classifier readout. EP uses the original hard-sigmoid energy dynamics, random beta sign,
-20 free steps, 4 nudged steps, layer rates `[0.1, 0.05]`, and persistent free particles for the fixed
-first 50k examples. This corrected EP is the strongest baseline and is also by far the most expensive.
+and persistent training particles on the fixed first 50k examples. Its published d1 schedule uses
+20 free steps, 4 nudged steps, `beta=0.5`, and layer rates `[0.1, 0.05]`; d2 uses 150 free steps,
+6 nudged steps, `beta=1`, and rates `[0.4, 0.1, 0.01]`. Evaluation starts fresh zero particles and
+runs the full free relaxation. The shallow EP protocol is the strongest native baseline here. The
+canonical deep protocol is much more expensive and highly initialization-sensitive in these runs.
 
 ### EP exact-architecture comparison
 
-All methods below use d1/w500, the same first 50k MNIST examples, and 25 epochs. EP retains its
-native raw-pixel energy protocol; BP/DFA/SDIL retain the shared z-score/tanh feedforward protocol.
-Forward parameter count is 397,510 for every row.
-
-| method | n | last test acc (%) | best test acc (%) | wall time (s) |
-|:-------|--:|------------------:|------------------:|--------------:|
-| BP | 2 | 98.190 ± 0.014 | 98.215 ± 0.007 | 15.1 ± 0.9 |
-| DFA | 2 | 97.620 ± 0.071 | 97.690 ± 0.099 | 9.9 ± 0.1 |
-| EP | 2 | 97.040 ± 0.269 | 97.220 ± 0.283 | 839.8 ± 78.4 |
-| SDIL | 2 | 97.510 ± 0.028 | 97.570 ± 0.028 | 15.4 ± 0.4 |
-
-SDIL is 0.47 points above EP on last-epoch mean and about 54.5x faster in wall time. This establishes
-that SDIL does not lose to the strongest baseline in the exact shallow architecture. It is **not**
-evidence of a unique scaling advantage: DFA is also strong on this easy d1 MNIST problem, and only
-two EP seeds have been run. At similar forward-parameter count but d3/w256, SDIL and EP are tied
-within noise (97.13% vs 97.04% last-epoch mean), again with a ~26x wall-time advantage for SDIL.
+All methods below use the same first 50k MNIST examples and the same epoch budget within each exact
+forward architecture: 25 epochs for d1/w500 (397,510 parameters) and 60 for d2/w500 (648,010
+parameters). EP retains its native raw-pixel hard-sigmoid energy protocol; BP/DFA/SDIL retain the
+shared z-score/tanh feedforward protocol.
+
+| depth | method | n | last test acc (%) | best test acc (%) | wall time (s) |
+|:------|:-------|--:|------------------:|------------------:|--------------:|
+| d1 | BP | 5 | 98.240 ± 0.073 | 98.272 ± 0.080 | 15.0 ± 0.5 |
+| d1 | DFA | 5 | 97.662 ± 0.078 | 97.700 ± 0.074 | 10.6 ± 1.6 |
+| d1 | EP | 5 | 96.926 ± 0.238 | 97.198 ± 0.147 | 800.5 ± 53.3 |
+| d1 | SDIL | 5 | 97.596 ± 0.089 | 97.638 ± 0.075 | 15.5 ± 0.2 |
+| d2 | BP | 5 | 98.288 ± 0.030 | 98.326 ± 0.036 | 45.1 ± 0.8 |
+| d2 | DFA | 5 | 97.590 ± 0.046 | 97.622 ± 0.036 | 34.6 ± 2.8 |
+| d2 | EP | 5 | 89.526 ± 4.461 | 90.028 ± 4.272 | 17530.4 ± 613.6 |
+| d2 | SDIL | 5 | 97.516 ± 0.067 | 97.562 ± 0.058 | 58.5 ± 4.0 |
+
+The paired SDIL−EP last-epoch gap is `0.670 ± 0.253` points at d1 and `7.990 ± 4.459` at d2;
+SDIL's geometric-mean wall-time speedup is 51.7x and 299.8x, respectively. The five d2 EP outcomes
+are 97.46%, 87.56%, 88.07%, 87.77%, and 86.77%. They are finite completed trajectories, not crashes:
+one seed reaches the shallow-level basin while four do not. This is evidence of initialization
+sensitivity under the reproduced canonical schedule, not evidence that EP is intrinsically
+unstable across implementations. Reproducing the basin split in the original Theano revision or a
+strong modern EP variant remains necessary. DFA is also strong on easy MNIST, so the scaling claim
+comes from the five-depth CIFAR panel; this comparison establishes that SDIL need not trade away
+accuracy to obtain a large cost advantage over canonical EP.
 
 ## How to run
-`experiments/run.py --mode {bp,dfa,sdil} --dataset {mnist,fmnist,cifar10} --depth D --residual {0,1} --act {tanh,gelu,silu,relu}`
+`experiments/run.py --mode {bp,fa,dfa,sdil} --dataset {mnist,fmnist,cifar10} --depth D --residual {0,1} --act {tanh,gelu,silu,relu}`
 Batteries: `experiments/run_v2.sh <ds> "<depths>" <res> <act> "<seeds>" <ep> <pfx>`.
-Canonical local baselines: `experiments/baseline_sweep.sh`; original d1 EP:
+Five-depth scaling: `experiments/figure_scaling_sweep.sh`; canonical local baselines:
+`experiments/baseline_sweep.sh`; original d1 EP:
 `experiments/ep_original_sweep.sh`; exact d1 comparison: `experiments/ep_archmatch_sweep.sh`;
 canonical d2 comparison: `experiments/ep_depth2_sweep.sh`; near-parameter comparison:
 `experiments/ep_matched_sweep.sh`.
-Audited aggregate tables: `experiments/analyze_verified.py`.
-Smoke/mechanism checks: `experiments/smoke.py` and `experiments/baseline_smoke.py`.
+Audited aggregate tables: `experiments/analyze_verified.py`. The claim-locked finalizer is
+`experiments/finalize_claims.sh`; it rejects missing/dirty inputs, regenerates both main figures,
+their captions, source-hash manifest, and `results/audited_tables.md`, then runs all smoke checks.
 
 ## Open items
 
@@ -184,9 +218,8 @@ Smoke/mechanism checks: `experiments/smoke.py` and `experiments/baseline_smoke.p
   not gain accuracy with depth.
 - Diagnose why residual FA has high directional cosine but lower accuracy: log update-norm ratios,
   parameter-space cosine, and serial feedback latency/cost.
-- Increase EP and exact-architecture comparisons from two to five seeds. If compute permits, run the
-  author's d2 EP schedule (60 epochs, 150 free steps, 6 nudged steps) rather than inventing a cheap
-  deep-EP approximation.
+- Reproduce the d2 EP basin sensitivity in the original Theano revision and/or a strong modern EP
+  variant without selectively removing seeds.
 - Tune SDIL schedules using a held-out validation split, never the test trajectory, then freeze the
   protocol before expanding the EP comparison.
 - Extend PEPITA and Forward-Forward beyond shallow MNIST before making any scaling claim about them.