diff options
Diffstat (limited to 'notes/README.md')
| -rw-r--r-- | notes/README.md | 91 |
1 files changed, 68 insertions, 23 deletions
diff --git a/notes/README.md b/notes/README.md index 76d8e46..ee608b6 100644 --- a/notes/README.md +++ b/notes/README.md @@ -1,37 +1,82 @@ # Notes -This folder collects working notes for the project tentatively titled: +Project memory for: **What Does Random Feedback Cost? (Feedback Alignment +capacity & operator-gap analysis)** — target AAAI-27. -**Distributional Capacity Bounds for Feedback Alignment in Multilayer Perceptrons** +## Start here (the writing pipeline) -Use this directory as the project memory. Keep rough ideas here before promoting them into a paper draft or experiments. +| doc | role | +|---|---| +| [41_paper_plan.md](41_paper_plan.md) | **the plan**: venue rules, narrative, section/page budget, theorem & figure numbering, related-work list, schedule, risks | +| [36_evidence_ledger.md](36_evidence_ledger.md) | **the numbers**: every claim with its corrected headline figure, claim discipline, booby traps | +| [40_reproduction_manifest.md](40_reproduction_manifest.md) | **the runs**: claim -> exact command -> output dir -> note; everything not listed is not citable | +| [37_wrapup_corrections.md](37_wrapup_corrections.md) | audit record: e0 sampler de-circularized; closed-form comparison matched (0.977 retired -> 0.933/0.982) | -## Start here (current state) +## Authoritative note per result -- [36_evidence_ledger.md](36_evidence_ledger.md): **the writing base** — full - evidence chain, corrected headline numbers, figure sources, claim discipline. -- [38_mnist_real_data_validation.md](38_mnist_real_data_validation.md): e0 - theorem + estimator replicated on MNIST (no fit). -- [39_teacher_test_gap.md](39_teacher_test_gap.md): teacher-task test gap — - optimization cost invariant, test effect task-dependent (sign flip). -- [37_wrapup_corrections.md](37_wrapup_corrections.md): pre-writing audit - fixes (e0 sampler now real FA backward; closed-form comparison now - data/init-matched — corr 0.977 retired, 0.933 matched / 0.982 ensemble). -- [35_current_results_consolidated.md](35_current_results_consolidated.md): contribution snapshot (numbers corrected per note 37). -- [34_closed_form_soft_ramp_theorem.md](34_closed_form_soft_ramp_theorem.md): closed-form gap law that quantifies the burden and explains the soft ramp. -- [18_contribution_roadmap.md](18_contribution_roadmap.md): contribution roadmap (includes the tangent-operator estimator). +| result | note | status | +|---|---|---| +| T1 Beta alignment law + capacity cost | 01 (statements/proofs), 02 (validation) | proven + validated | +| P1 multilayer additivity, Theta(Ln^2), Gamma(L,1) | 01, 02 | proven + validated | +| T2 prior-free minimax 1/D | 01, 02 | proven + validated | +| P2 random-subspace null model (and its honest failure as a predictor) | 26 (theorem), 25 (failure) | proven; overpredicts real FA | +| T3 initial erosion = hidden BP speed share | 29 | proven + validated (synthetic 0.4378/0.4403; MNIST note 38) | +| P3 exact Gaussian e0 (one hidden layer) | 32 + fix in 37 | proven + validated (real backward) | +| T4 closed-form soft-ramp gap law | 34 + correction in 37 | proven under A1+A2; corr 0.933 matched / 0.982 ensemble | +| early-velocity estimator (+ bracket, bias structure) | 15, 16, 17 | no-fit, snapshot-conditional; MAE 0.0019, corr 0.999 | +| dense soft ramp (refutes phase transition) | 22 | measured, 352 traj | +| finite-time validity chain (T=5 local; oracle K(t)) | 10, 13 | measured | +| MNIST real-data validation | 38 | measured | +| teacher-task test gap (sign flip) | 39 | measured | +| positioning (Song-Xu-Lafferty; effective-capacity lit; methods for K_t) | 27, 24, 30 | reference | +| proof-vs-estimator-vs-fit discipline | 28 | reference | +| appendix proof source material | 33 | reference (framing partly stale) | -## Files +## Historical chain (chronological; superseded items marked) -- [00_project_overview.md](00_project_overview.md): high-level thesis, contribution map, and scope. -- [01_theory_notes.md](01_theory_notes.md): notation, theorem candidates, proof sketches, and open theoretical gaps. -- [02_experiment_notes.md](02_experiment_notes.md): validation plan, metrics, datasets, and plotting ideas. -- [03_paper_outline.md](03_paper_outline.md): draft paper structure and section-level writing plan. +Phase 1 — static geometry (validated, still paper-grade): +- 00 project overview — **superseded framing** (banner added); 01 theory + notes — T1-T3 statements and proofs; 02 experiment notes — static + validation record (Beta/Gamma/minimax/overlap) + early trajectory probes; + 03 paper outline — **superseded** (banner added). -## Note Conventions +Phase 2 — capacity-margin era (Adam sweeps; superseded by SGD work): +- 04 downstream design; 05 random-task results; 06 peak-shift analysis — + all **superseded** by the Jacobian/NTK view of 07. + +Phase 3 — operator view and finite time: +- 07 BP capacity theory (the pivot to tangent operators); 08 SGD T=50 + validation (corr 0.97, 2x scale -> diagnosis); 09 fixed-width probe; + 10 **T=5 local exactness** (paper-grade); 11 finite-time roadmap; + 12 error decomposition (BP-too-strong/FA-too-weak); 13 **oracle K(t) + diagnostic** (paper-grade: drift is the whole error); 14 scalar predictors + fail (negative, kept); 15 **the estimator** (paper-grade); 16 **stress + grid** (paper-grade); 17 **bias structure** (paper-grade); 18 contribution + roadmap. + +Phase 4 — the phase-transition story dies: +- 19/20/21 kink-era analyses — **refuted** by longer training; 22 **dense + T=30000 soft ramp** (paper-grade); 23 soft-erosion revision. + +Phase 5 — exact theory of the burden: +- 24 literature + gap-theory positioning; 25 hard-k overprediction + (negative, kept); 26 null-model theorem + claim discipline; 27 + Song-Xu-Lafferty positioning; 28 proof-vs-fit status; 29 **initial-moment + theorem T3**; 30 methods survey for K_t dynamics; 31 derivative + extrapolation blows up (negative, kept); 32 **e0 Gaussian validation**; + 33 complete markdown draft (appendix source); 34 **closed-form law T4**; + 35 consolidated snapshot. + +Phase 6 — wrap-up for AAAI-27 (2026-06-09): +- 36 evidence ledger; 37 corrections; 38 **MNIST validation**; 39 **teacher + test gap**; 40 reproduction manifest; 41 paper plan. + +## Note conventions - Prefer formulas and precise claims over prose-only ideas. - Mark unproven statements as `Conjecture` or `Hypothesis`. -- Mark results ready for paper text as `Candidate theorem`, `Candidate proposition`, or `Candidate corollary`. +- Mark results ready for paper text as `Candidate theorem/proposition/corollary`. - Keep experiment ideas close to the metric they test. - When a note depends on an assumption, write the assumption explicitly. +- Never edit a historical note's results in place: append a dated correction + block and record the fix in a new note (as done in 34/35/37). |