blob: 2bdd7793861b7c3dafe37796a3aae83f45fdc454 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
|
# facap — What Does Random Feedback Cost?
Theory + experiments quantifying the training cost of **fixed random
feedback** relative to backpropagation. Exact initialization results cover
FA and DFA; finite-time analysis and experiments focus on FA. Target venue:
**AAAI-27** (abstract 2026-07-20, paper 2026-07-27).
## Results in one table
| claim | headline number |
|---|---|
| static alignment is an exact Beta(1/2,(D-1)/2) law; log-volume cost adds to Theta(Ln^2) | 700k samples, KS p 0.17-0.99; Gamma(L,1) max KS 0.0042 |
| no prior-free feedback initialization beats isotropic (minimax 1/D) | exact theorem + lam_min table |
| exact expected initial deficit = nonoutput BP decrease share, for FA and DFA | depth 1/2/3/4/6: max row error 0.0092; CE CNN max error 0.0083 |
| one-hidden-layer e0 is exactly Gaussian | KS p 0.13-0.999, real-backward sampler |
| random-direction deletion is not an FA cost model | up to 250x finite-time overprediction |
| matrix mismatch does not set FA's finite-time cost | depth 1->6: joint alignment proxy drops 18.1 orders while width-64 gap grows 0.0421->0.3461; frozen MAE 0.0669 vs two-snapshot 0.0104/0.0139 |
| initialization-rate compensation is material but incomplete | gap reduction 29-51% across depths; independently tuned BP/FA retain a gap |
| optimization cost is invariant; test-side effect is task-dependent | teacher task: test gap sign flip (-0.120+/-0.013 at w=32) |
## Layout
- `notes/` — project memory. Entry points: `notes/43_aaai_accept_bar_experiments.md`
(new experiments), `notes/41_paper_plan.md` (older writing plan),
`notes/36_evidence_ledger.md` (numbers), `notes/40_reproduction_manifest.md`
(runs). `notes/README.md` indexes all 43 notes with status.
- `scripts/` — all experiments (python, float64, CPU; deterministic seeds).
`scripts/README.md` has the status table and recipes.
- `outputs/` — gitignored, regenerable. Paper-grade runs are exactly those in
`notes/40_reproduction_manifest.md`; the rest is exploratory history.
- `paper/` — AAAI-27 scaffolding (`main.tex` needs AuthorKit 27;
`preview.tex` compiles standalone via tectonic). `body.tex` is being
rewritten from the evidence ledger.
## Setup
```bash
pip install -r requirements.txt # numpy scipy matplotlib torch (+ torchvision for MNIST)
python scripts/real_data_validation.py --self-test # sanity: kernels vs autograd
```
|
