path: root/notes/41_paper_plan.md

# AAAI-27 Paper Plan

The single planning document for the writing phase. Inputs: evidence ledger
(note 36), reproduction manifest (note 40), corrections (note 37). Supersedes
note 03 and the figure list in note 35.

## 0. Venue constraints (verified 2026-06-09)

- **AAAI-27**: abstract due **2026-07-20**, full paper **2026-07-27**
  (23:59:59 UTC-12); supplementary (technical appendix / code+data) due 3 days
  after the paper deadline. Conference 2027-02-16..23, Montreal.
- **7 pages of technical content + unlimited reference-only pages.**
- Reviewers are NOT required to read supplementary material => every
  load-bearing statement, number, and figure goes in the 7 pages; the appendix
  carries proofs and extended tables only.
- Mandatory reproducibility checklist; plan to submit code+configs as
  supplementary zip (manifest note 40 makes this mechanical).
- Format: AuthorKit 27 (`aaai27.sty`, not on CTAN — download required),
  anonymous submission.

## 1. Title

Working: **"What Does Random Feedback Cost? The Exact Price of Feedback
Alignment in MLPs"**

Alternates (decide at polish time):
- "What Does Random Feedback Cost? A Distributional Capacity and Operator-Gap
  Analysis of Feedback Alignment" (current scaffolding title; long)
- "The Soft Price of Random Feedback: Exact Capacity and Operator-Gap Laws for
  Feedback Alignment"

## 2. The pitch (frozen narrative)

A decade of FA work answered "can random feedback learn?" — we answer "what
does it cost?", exactly and with no fitted parameters. Three results: (i) the
static geometry of random feedback is an exact Beta law whose log-volume cost
adds to Theta(Ln^2), and (ii) no prior-free initialization can beat it
(minimax 1/D). (iii) The training cost it induces is *always nonzero, soft,
and computable*: at initialization the expected FA/BP speed gap equals exactly
the hidden-layer share of BP's learning speed (one BP backward pass predicts
it; verified on synthetic nets and MNIST); over training the gap follows a
closed-form spectral soft ramp (no phase transition — we refute the natural
redundancy-exhaustion story), and a two-snapshot early-velocity estimator
predicts the finite-time gap to ~2-4% with zero fitting. Capacity dilutes the
cost (k/P, lam_min growth); it never deletes it.

One-line summary: **random feedback is not free; its cost is irreducible but
soft, and now exactly priced.**

## 3. Section plan and page budget (7.00 pages)

| sec | pages | content (all numbers from notes 36/40 only) |
|---|---|---|
| 1 Introduction | 0.90 | cost-not-capability framing; the wrong natural guess (phase transition) and what replaces it; 3 contributions; explicit claim-scope sentence (optimization gap; estimator is snapshot-conditional) |
| 2 Setup | 0.55 | BP/FA backward, speeds, `e_0`, `K_BP`, `K_FA`, `S_FA`; loss convention `L=||r||^2/2N`; "output layer identical" fact |
| 3 Static cost (T1+P1) | 0.65 | Beta law (Thm 1, 2-line proof in main), log-volume cost, additivity + Theta(Ln^2) (Prop 1, proof appendix); validation sentence: 700k samples KS table + Gamma(L,1) max KS 0.0042 |
| 4 No free lunch at init (T3) | 0.40 | minimax Thm 2 + 3-line proof + the "anisotropy helps some directions only by hurting others" reading; empirical lam_min table sentence |
| 5 The gap | 2.30 | (a) null model Prop 2 (`Beta(k/2,(P-k)/2)`, `P(e=0)=0`) + **its honest 2x-250x overprediction** => motivates operator object; (b) Thm 3 `E_B[e_0]=hidden share` + proof + Gaussian Prop 3; MNIST share-0.39-0.82 confirmation sentence; (c) A1/A2 stated plainly, Thm 4 closed form + Cor 1 (soft, analytic, no threshold) + Cor 2 (ramp law `log gap ~ -c rho lam_min`); (d) the lazy chain in one paragraph WITH numbers (T=5 corr 0.99947 -> oracle K(t) MAE 0.000273 -> estimator) + estimator definition + bracket (velocity low / retangent high) + bias origin (drift-velocity decay) |
| 6 Experiments | 1.50 | F1-F4 + numbers table; soft ramp (refutation), closed form matched 0.933 / ensemble 0.982 / lam_min 0.987; e0 hidden-share calibration (max err <=0.008 over 6x6) + KS; estimator 256-traj MAE 0.00189 corr 0.99885 + stress grid range; MNIST both parts; teacher test-gap sign flip (2 sentences + pointer to F4/appendix) |
| 7 Related work | 0.45 | four paragraphs: FA empirics; FA theory (Song-Xu-Lafferty complementarity via `E_B[H_FA]=0`); NTK/lazy; capacity/memorization |
| 8 Limitations + conclusion | 0.25 | scope (MLP, full-batch, squared loss; MNIST-subset real data; conv/transformers open); estimator conditional; architecture-only drift = open problem |

Figure area is counted inside sections 5-6 (~1.6 pages of the budget).

## 4. Theorem numbering (frozen)

- **Thm 1** alignment Beta law; **Prop 1** multilayer additivity/scaling
- **Thm 2** prior-free minimax
- **Prop 2** random-subspace null model (erosion Beta, `E=k/P`)
- **Thm 3** initial erosion = hidden speed share; **Prop 3** exact Gaussian e0
  (one hidden layer)
- **Thm 4** closed-form gap (A1+A2); **Cor 1** soft/no-threshold; **Cor 2**
  ramp law
- Estimator: definition + empirical claims only — never call it a theorem.

## 5. Figures (4 main, AAAI two-column specs)

| # | panels | source (note 40 map) | placement |
|---|---|---|---|
| F1 | soft ramp vs closed form (log-y, error bars, matched + ensemble + band) ; ramp law vs lam_min | `closed_form_soft_ramp/` | sec 6, full width |
| F2 | e0 calibration: synthetic widths ; MNIST configs (share 0.39-0.82) | `actual_fa_initial_operator_moments/` + `real_data_validation_mnist/` | sec 5b/6, column |
| F3 | estimator: synthetic 256-traj scatter ; MNIST 96-traj scatter | `compressed_operator_*_plots/` + `real_data_validation_mnist/` | sec 6, full width |
| F4 | teacher task: train gap (soft) ; test gap (sign flip) | `teacher_test_gap_sgd_T8000/` | sec 6, column |

Appendix figures: static Beta/QQ + Gamma(L,1); e0 Gaussian panels; stress-grid
panels; T=5 horizon table-figure; oracle-K(t) diagnostic; estimator
trajectory band. All sources exist; re-render at column width with matched
fonts (week-2 task; one `paper/figures.py` script regenerating every figure
from the manifest runs, no manual edits).

## 6. Related work (citation list with one-line why)

FA empirics:
1. Lillicrap et al. 2016, Nat. Comms — FA works; alignment emerges. (origin)
2. Nokland 2016 — DFA in deep nets. (variant breadth)
3. Liao, Leibo, Poggio 2017 — sign symmetry; how important is weight symmetry. (initialization variants our Thm 2 prices)
4. Xiao et al. 2019 — sign-symmetry at ImageNet scale. (scaling empirics)
5. Bartunov et al. 2018 — bio-plausible methods stress-tested; FA degrades at scale. (the cost is visible empirically)
6. Launay et al. 2020 — DFA scales to modern architectures. (counterpoint; cost vs usability)
7. Akrout et al. 2019 — weight mirrors; Kolen-Pollack 1994 — learned transport. (ways to *pay* the cost rather than avoid it; contrast with fixed-B scope)
8. Crick 1989 / Grossberg 1987 — weight transport problem. (motivation, one cite)
9. Lillicrap et al. 2020, Nat Rev Neuro — backprop and the brain survey. (context)
10. Frenkel et al. 2021 — DRTP. (fixed-feedback family)

FA theory:
11. Song, Xu, Lafferty 2021 — two-layer FA convergence; `K_FA=G+H_FA`. (the complement: our Thm 3 = `E_B[H_FA]=0` in residual direction; convergence != operator equality — note 27 wording)
12. Refinetti, d'Ascoli, Ohana, Goldt 2021 — align-then-memorise dynamics. (their alignment phase = our drift `K_s-K_0`)
13. Saxe et al. 2014 — deep linear dynamics. (route to architecture-only theory, future work)

Kernels / lazy:
14. Jacot et al. 2018 — NTK. (the operator lens)
15. Lee et al. 2019 — wide nets as linear models. (lazy residual recursion)
16. Chizat, Oyallon, Bach 2019 — lazy training. (A1 scope)
17. Huang & Yau 2020 — neural tangent hierarchy. (the K_t-dynamics program our estimator instantiates empirically; note 30)

Capacity / memorization:
18. Zhang et al. 2017 — fitting random labels. (the "capacity absorbs it" intuition we refute softly)
19. Montanari & Zhong 2022 — interpolation phase transition. (why a kink was a reasonable guess)
20. (optional) Arora et al. 2019 exact NTK computation; cite only if space.

Rule: every citation earns a clause about how it relates to a *result* of
ours; no laundry lists. Bibliography moves to `aaai27.bst`-compatible `.bib`.

## 7. Writing rules (binding)

Prose-level discipline is governed by `paper/style_contract.md` (lexical bans,
em-dash budget, density floor, Buschek integrity mapping, and the
project-specific honesty rails). Run it as a per-section pass. The rules below
are the content-level commitments it assumes; the two files must agree.

1. Numbers only from notes 36/40; verify against the CSV when transcribing.
2. Never cite the retired 0.977; closed form = 0.933 matched / 0.982 ensemble.
3. Estimator = "snapshot-conditional, no fitted parameters", never
   architecture-only; state the lazy-regime scope (MNIST eta-sensitivity).
4. Hard-k appears only as a null model + reported failure.
5. Negative results (derivative blow-up MAE 270; scalar-gain corr 0.124) get
   one sentence each in sec 5d/appendix — they justify the estimator's form.
6. Every theorem states its assumptions inline (independence, zero-mean,
   output-layer-exact, A1/A2).
7. Prose style: every formula followed by a reading sentence; no unexplained
   jargon ("erosion", "speed" defined once in sec 2).
8. The teacher test-gap sign flip is reported as scoping evidence, not as a
   contribution claim.

## 8. Appendix plan (supplementary PDF)

A. Full proofs T1, P1, T2, P2, T3, P3 (source: note 33 sections; rewrite to
   match main-text notation), T4 + corollaries (note 34).
B. The finite-time chain: T=5 horizon table, oracle-K(t) diagnostic, bias
   decomposition + alpha_t curvature (notes 10/13/17).
C. Stress grid full table (note 16) + MNIST per-config table (note 38).
D. Negative results: hard-k overprediction table (25), derivative probe (31),
   scalar gain (14).
E. Experiment details: all configs/seeds (= manifest note 40), hardware,
   runtime; reproducibility checklist answers.
F. Teacher test-gap discussion + figure (note 39).

## 9. Schedule (today = Mon 2026-06-09)

| week | deliverable | exit criterion |
|---|---|---|
| 06-09..06-15 | wrap-up audit + this plan (**done**); download AuthorKit; `paper/figures.py` regenerating F1-F4 at AAAI specs | `main.tex` compiles with kit; 4 figures render |
| 06-16..06-22 | full body rewrite from ledger (sec 1-6), figures embedded; style_contract pass per section | 7-page draft, every number traced to manifest, contract pass logged |
| 06-23..06-29 | related work + `.bib`; appendix A-F; reproducibility checklist; code-release zip | complete submission package v1 |
| 06-30..07-06 | claims audit (read against ledger §9 + rules §7); number audit (against CSVs); full style_contract pass; external read if available | v2; no unverified number; contract clean |
| 07-07..07-13 | buffer for surprises; abstract polish; figure freeze | v3 frozen except typos |
| 07-14..07-20 | abstract registration (**07-20**); final proof pass | submitted abstract |
| 07-21..07-27 | paper submission (**07-27**); supplementary (+3 days) | submitted |

## 10. Risk register

| risk | mitigation |
|---|---|
| "MLPs/synthetic only" | MNIST validation in main text (F2/F3); limitations sentence: conv/transformers open |
| "train gap — who cares about generalization?" | F4 sign flip + scoping paragraph: optimization cost is the invariant; test effect task-dependent |
| "estimator is conditional, not a theory" | stated upfront in intro + limitations; closed form supplies the mechanism level; open-problem paragraph |
| novelty vs Song-Xu-Lafferty | complementarity paragraph via `E_B[H_FA]=0` (note 27); we quantify cost, they prove convergence |
| MNIST estimator corr 0.911 looks weak vs 0.999 | report MAE-first (3.9% relative); explain narrow gap range; per-config table in appendix |
| 7-page overflow | appendix offload order: stress-grid table -> e0 Gaussian panels -> teacher figure -> Prop 3 proof sketch |
| AuthorKit/compile friction | week-1 task, not week-4 |
| reviewer asks for per-mode rho_i or architecture-only drift | named open problem with a concrete route (NTK recursion / deep-linear), framed as future work |