summaryrefslogtreecommitdiff
path: root/notes/README.md
diff options
context:
space:
mode:
authorYurenHao0426 <Blackhao0426@gmail.com>2026-06-09 21:35:51 -0500
committerYurenHao0426 <Blackhao0426@gmail.com>2026-06-09 21:35:51 -0500
commitf860ba2ffb409dc1b7dfab9534ed05cb25692269 (patch)
treed30029bb8bb754a7c3c5f44fd13869e4e4178b84 /notes/README.md
parent84fa70be406248e660488ddc4c56ca5ea0034189 (diff)
Organize repo and add AAAI-27 writing plan
Root README orients the repo; notes/README reindexed by status (writing pipeline / authoritative note per result / six historical phases with superseded markers); scripts/README gains a status table. Note 40 = reproduction manifest (claim -> command -> outputs -> note; everything unlisted is not citable). Note 41 = the binding paper plan: AAAI-27 constraints (7pp + refs, appendix not guaranteed read), narrative, section/page budget, frozen theorem and figure numbering, related-work list with per-cite rationale, writing rules, weekly schedule to the 07-20/07-27 deadlines, and risk register. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Diffstat (limited to 'notes/README.md')
-rw-r--r--notes/README.md91
1 files changed, 68 insertions, 23 deletions
diff --git a/notes/README.md b/notes/README.md
index 76d8e46..ee608b6 100644
--- a/notes/README.md
+++ b/notes/README.md
@@ -1,37 +1,82 @@
# Notes
-This folder collects working notes for the project tentatively titled:
+Project memory for: **What Does Random Feedback Cost? (Feedback Alignment
+capacity & operator-gap analysis)** — target AAAI-27.
-**Distributional Capacity Bounds for Feedback Alignment in Multilayer Perceptrons**
+## Start here (the writing pipeline)
-Use this directory as the project memory. Keep rough ideas here before promoting them into a paper draft or experiments.
+| doc | role |
+|---|---|
+| [41_paper_plan.md](41_paper_plan.md) | **the plan**: venue rules, narrative, section/page budget, theorem & figure numbering, related-work list, schedule, risks |
+| [36_evidence_ledger.md](36_evidence_ledger.md) | **the numbers**: every claim with its corrected headline figure, claim discipline, booby traps |
+| [40_reproduction_manifest.md](40_reproduction_manifest.md) | **the runs**: claim -> exact command -> output dir -> note; everything not listed is not citable |
+| [37_wrapup_corrections.md](37_wrapup_corrections.md) | audit record: e0 sampler de-circularized; closed-form comparison matched (0.977 retired -> 0.933/0.982) |
-## Start here (current state)
+## Authoritative note per result
-- [36_evidence_ledger.md](36_evidence_ledger.md): **the writing base** — full
- evidence chain, corrected headline numbers, figure sources, claim discipline.
-- [38_mnist_real_data_validation.md](38_mnist_real_data_validation.md): e0
- theorem + estimator replicated on MNIST (no fit).
-- [39_teacher_test_gap.md](39_teacher_test_gap.md): teacher-task test gap —
- optimization cost invariant, test effect task-dependent (sign flip).
-- [37_wrapup_corrections.md](37_wrapup_corrections.md): pre-writing audit
- fixes (e0 sampler now real FA backward; closed-form comparison now
- data/init-matched — corr 0.977 retired, 0.933 matched / 0.982 ensemble).
-- [35_current_results_consolidated.md](35_current_results_consolidated.md): contribution snapshot (numbers corrected per note 37).
-- [34_closed_form_soft_ramp_theorem.md](34_closed_form_soft_ramp_theorem.md): closed-form gap law that quantifies the burden and explains the soft ramp.
-- [18_contribution_roadmap.md](18_contribution_roadmap.md): contribution roadmap (includes the tangent-operator estimator).
+| result | note | status |
+|---|---|---|
+| T1 Beta alignment law + capacity cost | 01 (statements/proofs), 02 (validation) | proven + validated |
+| P1 multilayer additivity, Theta(Ln^2), Gamma(L,1) | 01, 02 | proven + validated |
+| T2 prior-free minimax 1/D | 01, 02 | proven + validated |
+| P2 random-subspace null model (and its honest failure as a predictor) | 26 (theorem), 25 (failure) | proven; overpredicts real FA |
+| T3 initial erosion = hidden BP speed share | 29 | proven + validated (synthetic 0.4378/0.4403; MNIST note 38) |
+| P3 exact Gaussian e0 (one hidden layer) | 32 + fix in 37 | proven + validated (real backward) |
+| T4 closed-form soft-ramp gap law | 34 + correction in 37 | proven under A1+A2; corr 0.933 matched / 0.982 ensemble |
+| early-velocity estimator (+ bracket, bias structure) | 15, 16, 17 | no-fit, snapshot-conditional; MAE 0.0019, corr 0.999 |
+| dense soft ramp (refutes phase transition) | 22 | measured, 352 traj |
+| finite-time validity chain (T=5 local; oracle K(t)) | 10, 13 | measured |
+| MNIST real-data validation | 38 | measured |
+| teacher-task test gap (sign flip) | 39 | measured |
+| positioning (Song-Xu-Lafferty; effective-capacity lit; methods for K_t) | 27, 24, 30 | reference |
+| proof-vs-estimator-vs-fit discipline | 28 | reference |
+| appendix proof source material | 33 | reference (framing partly stale) |
-## Files
+## Historical chain (chronological; superseded items marked)
-- [00_project_overview.md](00_project_overview.md): high-level thesis, contribution map, and scope.
-- [01_theory_notes.md](01_theory_notes.md): notation, theorem candidates, proof sketches, and open theoretical gaps.
-- [02_experiment_notes.md](02_experiment_notes.md): validation plan, metrics, datasets, and plotting ideas.
-- [03_paper_outline.md](03_paper_outline.md): draft paper structure and section-level writing plan.
+Phase 1 — static geometry (validated, still paper-grade):
+- 00 project overview — **superseded framing** (banner added); 01 theory
+ notes — T1-T3 statements and proofs; 02 experiment notes — static
+ validation record (Beta/Gamma/minimax/overlap) + early trajectory probes;
+ 03 paper outline — **superseded** (banner added).
-## Note Conventions
+Phase 2 — capacity-margin era (Adam sweeps; superseded by SGD work):
+- 04 downstream design; 05 random-task results; 06 peak-shift analysis —
+ all **superseded** by the Jacobian/NTK view of 07.
+
+Phase 3 — operator view and finite time:
+- 07 BP capacity theory (the pivot to tangent operators); 08 SGD T=50
+ validation (corr 0.97, 2x scale -> diagnosis); 09 fixed-width probe;
+ 10 **T=5 local exactness** (paper-grade); 11 finite-time roadmap;
+ 12 error decomposition (BP-too-strong/FA-too-weak); 13 **oracle K(t)
+ diagnostic** (paper-grade: drift is the whole error); 14 scalar predictors
+ fail (negative, kept); 15 **the estimator** (paper-grade); 16 **stress
+ grid** (paper-grade); 17 **bias structure** (paper-grade); 18 contribution
+ roadmap.
+
+Phase 4 — the phase-transition story dies:
+- 19/20/21 kink-era analyses — **refuted** by longer training; 22 **dense
+ T=30000 soft ramp** (paper-grade); 23 soft-erosion revision.
+
+Phase 5 — exact theory of the burden:
+- 24 literature + gap-theory positioning; 25 hard-k overprediction
+ (negative, kept); 26 null-model theorem + claim discipline; 27
+ Song-Xu-Lafferty positioning; 28 proof-vs-fit status; 29 **initial-moment
+ theorem T3**; 30 methods survey for K_t dynamics; 31 derivative
+ extrapolation blows up (negative, kept); 32 **e0 Gaussian validation**;
+ 33 complete markdown draft (appendix source); 34 **closed-form law T4**;
+ 35 consolidated snapshot.
+
+Phase 6 — wrap-up for AAAI-27 (2026-06-09):
+- 36 evidence ledger; 37 corrections; 38 **MNIST validation**; 39 **teacher
+ test gap**; 40 reproduction manifest; 41 paper plan.
+
+## Note conventions
- Prefer formulas and precise claims over prose-only ideas.
- Mark unproven statements as `Conjecture` or `Hypothesis`.
-- Mark results ready for paper text as `Candidate theorem`, `Candidate proposition`, or `Candidate corollary`.
+- Mark results ready for paper text as `Candidate theorem/proposition/corollary`.
- Keep experiment ideas close to the metric they test.
- When a note depends on an assumption, write the assumption explicitly.
+- Never edit a historical note's results in place: append a dated correction
+ block and record the fix in a new note (as done in 34/35/37).