# Evidence Ledger (Writing Base for the AAAI Paper)

Authoritative consolidation of every result, number, and figure source in this
project, compiled directly from notes 00-35, the scripts, and the raw outputs.
**Write the paper from this file**, not from any earlier draft: the earlier
LaTeX draft was written without a careful pass over the experimental record and
omits much of the evidence chain below. Status of every claim is marked.

Numbers updated 2026-06-09 after the wrap-up corrections (note 37): the
closed-form comparison is now data/init-matched, and the e0-distribution
sampler runs the real FA backward pass.

---

## 0. Scope and claim discipline

- Setting: MLPs, squared loss `L = ||r||^2/(2N)`, full-batch training,
  feedback alignment = fixed random backward matrices `B_l` replacing
  `W_{l+1}^T`. Synthetic random-label / teacher tasks + MNIST-subset
  validation (note 38).
- The paper quantifies the **optimization (train-loss) gap** FA pays relative
  to BP. It does NOT claim FA fails to converge (compatible with Song-Xu-
  Lafferty), does not predict test accuracy, and the estimator is
  **conditional on two early kernel snapshots** — not architecture-only.
- "No fit" everywhere below means: no post-hoc scalar was tuned to match the
  reported data. Predictions are computed from initialization (and, for the
  estimator, one early snapshot).

## 1. Theorems (proof status)

| # | statement | status | source note |
|---|---|---|---|
| T1 | `Q = cos^2(a,b) ~ Beta(1/2,(D-1)/2)`, `E[Q]=1/D` for independent isotropic directions | exact | 00/01 |
| T2 | log-volume costs add: `C_all = sum_l C_l = Theta(L n^2)`; surprisal `S_l ~ Exp(1)`, `sum ~ Gamma(L,1)` | exact | 01 |
| T3 | prior-free minimax: `sup_mu inf_a E[(a^T b)^2] = 1/D`, attained by isotropic | exact | 01 |
| T4 | null model: Haar-random k-dim deletion gives `e(r) ~ Beta(k/2,(P-k)/2)`, `E[e]=k/P`, `P(e=0)=0` | exact (within null model) | 26 |
| T5 | **initial-erosion theorem**: `E_B[e_0 | W,r] = 1 - ||g_out^BP||^2 / sum_l ||g_l^BP||^2` (hidden speed share), any depth, any independent zero-mean feedback | exact | 29 |
| T6 | one-hidden-layer Gaussian FA: `e_0 | W,r` exactly Gaussian with explicit mean/std (linear in B) | exact | 32 |
| T7 | closed-form gap under A1 (lazy) + A2 (scalar erosion `S_FA ~= rho K_BP`): `gap_T = (1/2N) sum_i c_i^2 [(1-eta rho lam_i/N)^{2T} - (1-eta lam_i/N)^{2T}]`; analytic => soft ramp, no threshold unless `rho -> 0`; converged regime `log gap_T ~= const - (2 eta T/N) rho lam_min` | exact under A1+A2 | 34 |

A2 is exact in the residual direction by T5 (`rho = 1 - E_B[e_0]` = output
share); extending it to all eigenmodes is the one modeling step. A1 is the
standard lazy approximation whose breakdown is *measured* (item 3c below).

## 2. Static geometry: validation record

| claim | evidence | numbers | output dir |
|---|---|---|---|
| Beta law | 700k samples, D = 64..4096 | KS stat 0.002-0.004, p 0.17-0.99; mean/q99 match to 4 digits | `capacity_empirical_validation` |
| capacity tail cost | same run | max smoothed cost error 0.20 nats, mean 0.019; multilayer product max err 0.46 nats | same |
| `Gamma(L,1)` surprisal | 100k per (D,L), D in {64..4096}, L in {1..16} | max KS **0.0042**, mean abs mean-error 0.0042 | `multilayer_capacity_distribution` |
| minimax | 5 schemes, D=32/128 | isotropic lam_min 0.0291 vs 1/32=0.0313; rademacher 0.0289; anisotropic 0.0061; subspace/axis 0 | `minimax_initialization`, `initialization_distribution_matching` |
| functional overlap | P=96,d=24, trials=100 | hard loss exact (0 until k>P-d, then k-(P-d)); soft overlap kd/P to 3 digits | `functional_capacity_overlap` |

## 3. The gap pillar: four-level evidence chain

### (a) Null model + its honest failure (negative result, keep in paper)

- T4 proves nonzero soft cost in the cleanest model; `P(e=0)=0`.
- **Hard-k overpredicts real FA**: predicted gap 0.040 at margin +130 vs
  measured 0.00016 (250x); ~2x on local erosion (measured local erosion 0.23-
  0.39 vs Beta-mean 0.63-0.73) (note 25, `soft_erosion_theory_vs_empirical`).
  Role in paper: motivates the *operator* object; hard constraint count is
  not the effective burden.

### (b) Exact at t=0 (T5 + T6)

- Mean theorem, 2-hidden-layer, real FA backward, 6 widths x 6 inits x 512
  draws: `E_B[e_0] = 0.4378 +/- 0.008` vs theory `0.4403`; per-width errors
  0.0003-0.008 (note 29, `actual_fa_initial_operator_moments`).
- Independent t=0 anchor from the (superseded) recovery probe: 0.4026+/-0.0066
  vs 0.4071 (run log, note 37).
- Full distribution (T6), **sampler = real FA backward after note-37 fix**:
  4 widths x 3 inits x 4096 draws, KS p in [0.13, 0.999], all means/stds match
  to 3-4 digits; linear identity `<C,B>` verified against the real backward to
  max err 8.9e-16 (`actual_fa_initial_erosion_distribution`).
- **MNIST (note 38)**: 16 configs (width {64,256} x depth {1,2} x 4 inits,
  512 real-backward draws each): hidden share spans **0.39-0.82** and
  `E_B[e_0]` lands on it with |diff| 0.0001-0.0089 (all within ~2 stderr) —
  the theorem holds on real data with no fit
  (`real_data_validation_mnist/e0_mnist_rows.csv`).

### (c) Finite time: the lazy chain (previous draft omitted ALL of this)

1. **T=5 local exactness** (note 10): frozen K(0), 256 traj, 8 N values:
   corr **0.99947**, MAE **0.00083**, mean ratio 0.993. Horizon table: ratio
   1.028/0.987/0.942/0.884/0.814 at T=1/5/10/20/50. => the operator object is
   right; freezing is a *local* theorem.
2. **Drift is the whole error** (note 13): at T=50, oracle time-varying K(t)
   product cuts gap MAE 0.0277 -> **0.000273** (100x); BP loss MAE 0.0254 ->
   0.00024. => no higher-order Taylor mystery; model K_t and you are done.
3. **Estimator** `K_hat_t = K_0 + (t/s)(K_s - K_0)` (notes 15-17): 256 traj,
   T=50, s=20: MAE **0.00189**, bias -0.0015, corr **0.99885**; fixed-K(0)
   MAE 0.0185 (10x worse); re-tangent MAE 0.0049 biased high => **no-fit
   bracket** `linear <= empirical <= retangent`.
   Stress grid (7 settings): depth 1/2/3, width 32/64/96, horizon 25/50/100:
   linear MAE 0.00023-0.0057, corr 0.994-0.99989
   (`operator_stress_grid_summary`).
4. **Bias structure** (note 17): residual -0.0021 gap bias = BP-loss
   overprediction (+0.0026), FA near-unbiased (+0.0005); cause: drift velocity
   decays after s (alpha_t < t/s; e.g. true alpha_50 = 1.46 (BP) / 1.17 (FA)
   vs linear 2.5). Honest error story, no fitted correction.
5. **MNIST replication (note 38)**: same protocol, widths {128,256} x
   N {128,256,512} x 2 inits x 8 feedbacks = 96 traj, lazy step
   (`eta lam_max/N = 0.05`): velocity MAE **0.00196** (= 3.9% of mean gap),
   bias -0.0006, corr 0.911 (gap range only 0.040-0.063, so MAE/bias are the
   operative stats); fixed K(0) MAE 0.0061 biased high — same bracket as
   synthetic. At eta lam_max/N = 0.3 the lazy chain breaks as expected
   (scope boundary, recorded in note 38).

### (d) The soft ramp + closed form (T7)

- Dense sweep (note 22): T=30000, 11 widths (20..40), 352 traj: gap
  0.1468 -> 0.000162 across margin -190 -> +130, smooth, log-linear
  (`log gap ~ -0.0209*margin`, R^2 **0.993**), **no kink at margin 0,
  nonzero at positive margin** => refutes the redundancy-exhaustion phase
  transition (and notes 19-21 document how the kink hypothesis died as T grew:
  near-margin-0 gap 0.297 at T=3000, 0.051 at T=10^4, 0.0037 at T=3x10^4).
- Closed form vs measured (**matched data + init after note-37 fix**):
  - matched single init: corr(log,log) = **0.933**;
  - 5-init geometric mean (same data): corr **0.982**; ensemble-mean
    lam_min(w) rises 0.026 -> 0.763 and explains the ramp at corr **0.987**;
    rho stable 0.68-0.79 across widths.
  - the previously reported corr 0.977 came from a comparison computed on the
    wrong data seed and init seed; retire that number (note 37).
  - frozen-init compresses the range (under-predicts large-gap end, over-
    predicts small-gap tail) — exactly the drift the estimator corrects;
    state this as mechanism-vs-magnitude.
  (`closed_form_soft_ramp/closed_form_vs_measured.csv`, `closed_form_per_seed.csv`)

## 4. Negative results worth reporting (reviewer-proofing)

| failed approach | number | note |
|---|---|---|
| hard-k as gap predictor | overpredicts 2x-250x | 25 |
| infinitesimal derivative extrapolation `K_0 + t dK_0` | MAE **270** vs 0.024 baseline (diverges) | 31 |
| scalar directional-gain model across N | corr 0.124 (vs operator 0.999) | 14 |
| trajectory bridge on long Adam horizons | shape corr 0.97 but 2-3x scale error | 02/08 |
| phase-transition (kink) hypothesis | dissolves as T grows; dense T=30000 smooth | 19-22 |

These define why the final objects (operator erosion + early velocity) are
what they are; (31) and (25) deserve explicit mention in the paper.

## 5. Real-data and generalization coverage (new, note 38)

- MNIST e0 + estimator: `outputs/real_data_validation_mnist/` (script
  `scripts/real_data_validation.py`, self-test verifies kernels vs autograd
  at 1e-15).
- Teacher-task test gap (note 39, SGD, widths 8..96, 24 FA runs/width):
  train gap soft and monotone (0.249 -> 0.0069) while the **test gap changes
  sign** — +0.177 (w=8), **-0.120 +/- 0.013 (w=32, FA generalizes better)**,
  +0.045 (w=96). The optimization gap is the invariant cost; its test-side
  consequence is task-dependent — exactly why the theory is built on the
  former. `outputs/teacher_test_gap_sgd_T8000/teacher_train_vs_test_gap.png`.

## 6. Figure plan (sources verified on disk)

| fig | content | source |
|---|---|---|
| F1 | static law: Beta hist + QQ (+ Gamma(L,1) inset or appendix) | `capacity_empirical_validation` |
| F2 | dense soft ramp + closed form (matched + 5-init mean + band) + lam_min ramp law | `closed_form_soft_ramp/closed_form_vs_measured_ramp.png` |
| F3 | e0: moment calibration + Gaussian distribution panels (real backward) | `actual_fa_initial_operator_moments`, `actual_fa_initial_erosion_distribution` |
| F4 | estimator: predicted-vs-empirical scatter, 256 traj + stress grid summary | `compressed_operator_s20_256traj_T50_width64_plots`, `operator_stress_grid_summary` |
| F5 | MNIST: e0 calibration + estimator scatter (real data) | `real_data_validation_mnist` |
| F6 (optional/appendix) | T=5 exactness + oracle-K(t) diagnostic (the lazy chain) | notes 10/13 outputs |

## 7. Main text vs appendix split

- Main: T1-T3 (brief), T4 + failure (a), T5 (centerpiece theorem), T7 +
  Corollaries, estimator + stress grid, dense ramp, MNIST.
- Appendix: full proofs (T1-T7 from note 33 + 34), T6 derivation + KS table,
  the finite-time evidence chain details (T=5 table, oracle diagnostic, bias
  decomposition, alpha_t curvature), negative results (31, 25), experiment
  configs/seeds, teacher test-gap figure.

## 8. Open problems (state, do not solve)

Architecture-only finite-time theory: predict `rho` and the drift `K_s - K_0`
from initialization statistics (infinite-width recursions / deep-linear order
parameters); per-mode spectral erosion `rho_i` refining A2. (note 35)

## 9. Known booby traps for the writing pass

- Do not cite corr 0.977 for the closed form (mismatched; see note 37).
- Do not present the estimator as architecture-only; it conditions on (K_0,K_s).
- Do not present hard-k as a gap predictor; it is the null model only.
- `E[e]=k/P` and `rho` are different objects (null model vs measured share).
- The e0-distribution figure must be the post-fix (real-backward) rerun.
- Dense-run measured gaps live in `width_summary.csv` (train_gap_mean), 1 init
  x 32 feedback seeds — say so in the caption.
- All speeds/losses use `L = ||r||^2/(2N)`; `e_0` is defined from speeds, not
  losses.