1 files changed, 141 insertions, 0 deletions
diff --git a/notes/00_project_overview.md b/notes/00_project_overview.md
new file mode 100644
index 0000000..3f8696e
--- /dev/null
+++ b/notes/00_project_overview.md
@@ -0,0 +1,141 @@
+# Project Overview
+
+## Working Title
+
+Distributional Capacity Bounds for Feedback Alignment in Multilayer Perceptrons
+
+## Core Question
+
+Feedback alignment (FA) replaces the exact BP feedback operator with a fixed random matrix. The central question is:
+
+> Can we quantify, distributionally and architecturally, how much additional alignment burden FA introduces relative to BP?
+
+The goal is not to claim that architecture alone predicts final test accuracy. The goal is to derive distributional bounds and capacity proxies for the extra alignment burden induced by random feedback, then validate how these proxies relate to observed FA/BP trajectory gaps.
+
+## Contribution Map
+
+### 1. Capacity Formalization
+
+Define a hierarchy of capacity notions for FA:
+
+- Angular alignment statistic.
+- Log-volume parameter capacity cost.
+- Task-projected functional capacity.
+
+Layerwise alignment:
+
+\[
+Q_l =
+\cos^2(W_{l+1}^{\top}, B_l)
+=
+\frac{
+\langle W_{l+1}^{\top}, B_l\rangle_F^2
+}{
+\|W_{l+1}^{\top}\|_F^2 \|B_l\|_F^2
+}.
+\]
+
+If directions are independent and isotropic in dimension \(D_l=n_l n_{l+1}\):
+
+\[
+Q_l \sim \mathrm{Beta}\left(\frac12,\frac{D_l-1}{2}\right).
+\]
+
+Define log-volume cost:
+
+\[
+C_l(q) =
+-\log \Pr(Q_l\ge q)
+=
+-\log\left[
+1-I_q\left(\frac12,\frac{D_l-1}{2}\right)
+\right].
+\]
+
+### 2. Scaling Law and Redundancy Exhaustion
+
+Across independent layers:
+
+\[
+p_{\mathrm{all}} = \prod_l p_l(q_l),
+\qquad
+C_{\mathrm{all}} = -\log p_{\mathrm{all}} = \sum_l C_l(q_l).
+\]
+
+Therefore:
+
+- Log-capacity cost accumulates linearly.
+- Raw feasible volume decays geometrically.
+
+For equal-width MLPs with width \(n\), depth \(L\), and fixed threshold \(q\):
+
+\[
+C_{\mathrm{all}} = \Theta(Ln^2),
+\qquad
+p_{\mathrm{all}} = \exp[-\Theta(Ln^2)].
+\]
+
+Functional gap should appear when alignment constraints exhaust redundant directions. With total parameter dimension \(P\), local task rank \(d=\operatorname{rank}(J)\), and effective alignment constraint rank \(k\):
+
+\[
+\Delta d_{\mathrm{hard}}
+=
+\max(0, k-(P-d)).
+\]
+
+### 3. Prior-Free Minimax Bound
+
+For normalized feedback direction:
+
+\[
+\hat b = \frac{\operatorname{vec}(B)}{\|B\|_F}\in \mathbb S^{D-1},
+\]
+
+any initialization distribution \(\mu\) induces:
+
+\[
+M_\mu = \mathbb E_{\mu}[\hat b\hat b^\top],
+\qquad
+\operatorname{tr} M_\mu = 1.
+\]
+
+For unknown target direction \(a\):
+
+\[
+\mathbb E_\mu[(a^\top \hat b)^2] = a^\top M_\mu a.
+\]
+
+Thus:
+
+\[
+\sup_\mu \inf_{\|a\|=1}
+\mathbb E_\mu[(a^\top \hat b)^2]
+=
+\frac1D.
+\]
+
+Isotropic random feedback achieves this minimax optimum. Scale, orthogonality, sparsity, and low rank can affect conditioning or cost, but cannot break the prior-free angular bound.
+
+### 4. Distributional Validation
+
+Validate:
+
+- The beta law for static alignment.
+- Capacity scaling across width, depth, rank, sparsity, and threshold.
+- Redundancy-exhaustion transition.
+- Relationship between capacity proxies and trajectory-level FA/BP gaps.
+
+## Scope Control
+
+Initial scope:
+
+- MLPs.
+- Dense FA first.
+- Gaussian or isotropic feedback first.
+- Synthetic regression and simple classification benchmarks.
+
+Defer:
+
+- CNNs, Transformers, BatchNorm, residual networks.
+- Strong claims about final test accuracy.
+- Biologically detailed interpretations.