diff options
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 20 |
1 files changed, 11 insertions, 9 deletions
@@ -1,9 +1,9 @@ # facap — What Does Random Feedback Cost? -Theory + experiments quantifying the training cost of **feedback alignment** -(FA: fixed random backward matrices instead of transposed weights) relative to -backpropagation in MLPs. Target venue: **AAAI-27** (abstract 2026-07-20, -paper 2026-07-27). +Theory + experiments quantifying the training cost of **fixed random +feedback** relative to backpropagation. Exact initialization results cover +FA and DFA; finite-time analysis and experiments focus on FA. Target venue: +**AAAI-27** (abstract 2026-07-20, paper 2026-07-27). ## Results in one table @@ -11,17 +11,19 @@ paper 2026-07-27). |---|---| | static alignment is an exact Beta(1/2,(D-1)/2) law; log-volume cost adds to Theta(Ln^2) | 700k samples, KS p 0.17-0.99; Gamma(L,1) max KS 0.0042 | | no prior-free feedback initialization beats isotropic (minimax 1/D) | exact theorem + lam_min table | -| initial FA/BP speed gap = hidden-layer BP speed share, exactly, any depth | synthetic 0.4378 vs 0.4403; MNIST diff <= 0.0089 over share 0.39-0.82 | +| exact expected initial deficit = nonoutput BP decrease share, for FA and DFA | depth 1/2/3/4/6: max row error 0.0092; CE CNN max error 0.0083 | | one-hidden-layer e0 is exactly Gaussian | KS p 0.13-0.999, real-backward sampler | -| the finite-time gap is a closed-form **soft ramp** — no phase transition | dense T=30000: log-linear R^2 0.993; closed form corr 0.933 matched / 0.982 ensemble; lam_min corr 0.987 | -| a two-snapshot early-velocity estimator predicts the gap with no fit | 256 traj: MAE 0.0019, corr 0.999; stress grid 0.994-0.99989; MNIST MAE 0.0020 (3.9% rel) | +| random-direction deletion is not an FA cost model | up to 250x finite-time overprediction | +| matrix mismatch does not set FA's finite-time cost | depth 1->6: joint alignment proxy drops 18.1 orders while width-64 gap grows 0.0421->0.3461; frozen MAE 0.0669 vs two-snapshot 0.0104/0.0139 | +| initialization-rate compensation is material but incomplete | gap reduction 29-51% across depths; independently tuned BP/FA retain a gap | | optimization cost is invariant; test-side effect is task-dependent | teacher task: test gap sign flip (-0.120+/-0.013 at w=32) | ## Layout -- `notes/` — project memory. Entry points: `notes/41_paper_plan.md` (plan), +- `notes/` — project memory. Entry points: `notes/43_aaai_accept_bar_experiments.md` + (new experiments), `notes/41_paper_plan.md` (older writing plan), `notes/36_evidence_ledger.md` (numbers), `notes/40_reproduction_manifest.md` - (runs). `notes/README.md` indexes all 41 notes with status. + (runs). `notes/README.md` indexes all 43 notes with status. - `scripts/` — all experiments (python, float64, CPU; deterministic seeds). `scripts/README.md` has the status table and recipes. - `outputs/` — gitignored, regenerable. Paper-grade runs are exactly those in |