README.md


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

# facap — What Does Random Feedback Cost?

Theory + experiments quantifying the training cost of **feedback alignment**
(FA: fixed random backward matrices instead of transposed weights) relative to
backpropagation in MLPs. Target venue: **AAAI-27** (abstract 2026-07-20,
paper 2026-07-27).

## Results in one table

| claim | headline number |
|---|---|
| static alignment is an exact Beta(1/2,(D-1)/2) law; log-volume cost adds to Theta(Ln^2) | 700k samples, KS p 0.17-0.99; Gamma(L,1) max KS 0.0042 |
| no prior-free feedback initialization beats isotropic (minimax 1/D) | exact theorem + lam_min table |
| initial FA/BP speed gap = hidden-layer BP speed share, exactly, any depth | synthetic 0.4378 vs 0.4403; MNIST diff <= 0.0089 over share 0.39-0.82 |
| one-hidden-layer e0 is exactly Gaussian | KS p 0.13-0.999, real-backward sampler |
| the finite-time gap is a closed-form **soft ramp** — no phase transition | dense T=30000: log-linear R^2 0.993; closed form corr 0.933 matched / 0.982 ensemble; lam_min corr 0.987 |
| a two-snapshot early-velocity estimator predicts the gap with no fit | 256 traj: MAE 0.0019, corr 0.999; stress grid 0.994-0.99989; MNIST MAE 0.0020 (3.9% rel) |
| optimization cost is invariant; test-side effect is task-dependent | teacher task: test gap sign flip (-0.120+/-0.013 at w=32) |

## Layout

- `notes/` — project memory. Entry points: `notes/41_paper_plan.md` (plan),
  `notes/36_evidence_ledger.md` (numbers), `notes/40_reproduction_manifest.md`
  (runs). `notes/README.md` indexes all 41 notes with status.
- `scripts/` — all experiments (python, float64, CPU; deterministic seeds).
  `scripts/README.md` has the status table and recipes.
- `outputs/` — gitignored, regenerable. Paper-grade runs are exactly those in
  `notes/40_reproduction_manifest.md`; the rest is exploratory history.
- `paper/` — AAAI-27 scaffolding (`main.tex` needs AuthorKit 27;
  `preview.tex` compiles standalone via tectonic). `body.tex` is being
  rewritten from the evidence ledger.

## Setup

```bash
pip install -r requirements.txt   # numpy scipy matplotlib torch (+ torchvision for MNIST)
python scripts/real_data_validation.py --self-test   # sanity: kernels vs autograd
```