diff options
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 38 |
1 files changed, 38 insertions, 0 deletions
diff --git a/README.md b/README.md new file mode 100644 index 0000000..171bd24 --- /dev/null +++ b/README.md @@ -0,0 +1,38 @@ +# facap — What Does Random Feedback Cost? + +Theory + experiments quantifying the training cost of **feedback alignment** +(FA: fixed random backward matrices instead of transposed weights) relative to +backpropagation in MLPs. Target venue: **AAAI-27** (abstract 2026-07-20, +paper 2026-07-27). + +## Results in one table + +| claim | headline number | +|---|---| +| static alignment is an exact Beta(1/2,(D-1)/2) law; log-volume cost adds to Theta(Ln^2) | 700k samples, KS p 0.17-0.99; Gamma(L,1) max KS 0.0042 | +| no prior-free feedback initialization beats isotropic (minimax 1/D) | exact theorem + lam_min table | +| initial FA/BP speed gap = hidden-layer BP speed share, exactly, any depth | synthetic 0.4378 vs 0.4403; MNIST diff <= 0.0089 over share 0.39-0.82 | +| one-hidden-layer e0 is exactly Gaussian | KS p 0.13-0.999, real-backward sampler | +| the finite-time gap is a closed-form **soft ramp** — no phase transition | dense T=30000: log-linear R^2 0.993; closed form corr 0.933 matched / 0.982 ensemble; lam_min corr 0.987 | +| a two-snapshot early-velocity estimator predicts the gap with no fit | 256 traj: MAE 0.0019, corr 0.999; stress grid 0.994-0.99989; MNIST MAE 0.0020 (3.9% rel) | +| optimization cost is invariant; test-side effect is task-dependent | teacher task: test gap sign flip (-0.120+/-0.013 at w=32) | + +## Layout + +- `notes/` — project memory. Entry points: `notes/41_paper_plan.md` (plan), + `notes/36_evidence_ledger.md` (numbers), `notes/40_reproduction_manifest.md` + (runs). `notes/README.md` indexes all 41 notes with status. +- `scripts/` — all experiments (python, float64, CPU; deterministic seeds). + `scripts/README.md` has the status table and recipes. +- `outputs/` — gitignored, regenerable. Paper-grade runs are exactly those in + `notes/40_reproduction_manifest.md`; the rest is exploratory history. +- `paper/` — AAAI-27 scaffolding (`main.tex` needs AuthorKit 27; + `preview.tex` compiles standalone via tectonic). `body.tex` is being + rewritten from the evidence ledger. + +## Setup + +```bash +pip install -r requirements.txt # numpy scipy matplotlib torch (+ torchvision for MNIST) +python scripts/real_data_validation.py --self-test # sanity: kernels vs autograd +``` |
