Add complete markdown draft

1 files changed, 1134 insertions, 0 deletions

diff --git a/notes/33_complete_markdown_draft.md b/notes/33_complete_markdown_draft.md
new file mode 100644
index 0000000..2a9e8da
--- /dev/null
+++ b/notes/33_complete_markdown_draft.md
@@ -0,0 +1,1134 @@
+# Distributional Capacity Bounds for Feedback Alignment in Multilayer Perceptrons
+
+Status: integrated Markdown draft for the current project.
+
+This document consolidates the clean version of the theory and experiments. It
+uses plain-text formulas instead of TeX.
+
+## One-Sentence Claim
+
+Random feedback alignment introduces a quantifiable distributional operator
+burden relative to BP. This burden can be exactly characterized at the level of
+random matrix alignment and at the initial FA operator level, and prior-free
+feedback initialization cannot beat the isotropic random baseline in worst-case
+angular alignment.
+
+## Main Contributions
+
+1. **Distributional capacity formalization.** We define feedback-alignment
+   capacity through matrix-direction alignment and log-volume cost, and derive
+   the exact Beta law for random feedback alignment.
+
+2. **Capacity scaling law.** We prove that raw feasible volume multiplies
+   across independent feedback blocks while log-capacity cost adds. For
+   equal-width MLPs, fixed-threshold log-cost scales like `Theta(L n^2)`.
+
+3. **Prior-free minimax feedback initialization.** We prove that without prior
+   information about the target backward direction, no initialization can beat
+   the `1/D` worst-case expected squared-alignment bound. Isotropic random
+   feedback achieves the bound.
+
+4. **Actual FA initial operator erosion.** We prove that the true initial FA
+   operator burden is not the raw feedback-matrix constraint count. Conditional
+   on fixed forward weights and residuals, the expected FA learning-speed loss
+   equals the hidden-layer BP speed share. In one-hidden-layer Gaussian FA, the
+   entire conditional erosion distribution is Gaussian.
+
+5. **Distributional empirical validation.** We validate the theory by sampling
+   large ensembles of random feedback matrices. The empirical matrix-alignment
+   and initial-operator-erosion distributions match the predicted distributions
+   without fitted parameters.
+
+## What This Paper Does Not Claim
+
+Do not claim:
+
+```text
+FA cannot converge in overparameterized networks.
+```
+
+This is contradicted by existing overparameterized FA theory in restricted
+settings.
+
+Do not claim:
+
+```text
+hard capacity margin produces a sharp FA/BP phase transition.
+```
+
+Our long-training sweeps showed a soft ramp, not a hard kink.
+
+Do not claim:
+
+```text
+architecture alone predicts all finite-time FA/BP loss gaps.
+```
+
+Finite-time gap prediction requires time-dependent operator dynamics. We have
+useful tangent-operator estimators, but they are not part of the main theorem
+stack in this cleaned version.
+
+## Setup
+
+Consider an MLP with widths:
+
+```text
+n_0, n_1, ..., n_L
+```
+
+Forward weights:
+
+```text
+W_l in R^{n_l x n_{l-1}}
+```
+
+For ordinary FA, the feedback matrix replacing the BP backward map from layer
+`l+1` to layer `l` is:
+
+```text
+B_l in R^{n_l x n_{l+1}}
+```
+
+The corresponding BP backward matrix is:
+
+```text
+A_l = W_{l+1}^T
+```
+
+with matrix-direction dimension:
+
+```text
+D_l = n_l n_{l+1}
+```
+
+For squared loss on a training set:
+
+```text
+L = ||r||^2 / (2N)
+```
+
+where:
+
+```text
+r = f_theta(X) - y
+```
+
+Let `g_l^BP` be the BP gradient block and `g_l^FA` be the FA pseudo-gradient
+block at layer `l`.
+
+Define the BP local learning speed:
+
+```text
+speed_BP = sum_l ||g_l^BP||^2
+```
+
+and the FA first-order local learning speed:
+
+```text
+speed_FA = sum_l <g_l^BP, g_l^FA>
+```
+
+The initial FA operator erosion is:
+
+```text
+e_0(B) = 1 - speed_FA / speed_BP
+```
+
+This is the one-step fractional loss of BP learning speed caused by using FA
+instead of BP.
+
+## Theorem 1: Random Feedback Alignment Has an Exact Beta Law
+
+### Statement
+
+Let:
+
+```text
+a = vec(A) / ||A||_F
+b = vec(B) / ||B||_F
+```
+
+where `a` and `b` are independent isotropic directions in `R^D`.
+
+Define squared alignment:
+
+```text
+Q = (a^T b)^2
+```
+
+Then:
+
+```text
+Q ~ Beta(1/2, (D - 1)/2)
+```
+
+Equivalently:
+
+```text
+P(Q <= q) = I_q(1/2, (D - 1)/2)
+```
+
+where `I_q` is the regularized incomplete beta function.
+
+For layer `l`:
+
+```text
+D_l = n_l n_{l+1}
+Q_l = cos^2(W_{l+1}^T, B_l)
+Q_l ~ Beta(1/2, (D_l - 1)/2)
+```
+
+High-dimensional approximation:
+
+```text
+D Q -> chi-square_1
+E[Q] = 1/D
+typical |cos angle| = O(D^{-1/2})
+```
+
+### Capacity Cost
+
+For an alignment threshold `q`, define:
+
+```text
+p(q) = P(Q >= q)
+C(q) = -log p(q)
+```
+
+Using the Beta law:
+
+```text
+C(q) = -log [1 - I_q(1/2, (D - 1)/2)]
+```
+
+For small `q`:
+
+```text
+C(q) approximately (D q) / 2
+```
+
+For fixed `q` in high dimension:
+
+```text
+C(q) approximately ((D - 1)/2) log(1/(1-q))
+```
+
+### Proof Sketch
+
+By rotational invariance, fix `a = e_1`. Then:
+
+```text
+Q = b_1^2
+```
+
+where `b` is uniform on the unit sphere in `R^D`. The squared first coordinate
+of a uniform sphere vector follows:
+
+```text
+Beta(1/2, (D - 1)/2)
+```
+
+This gives the result.
+
+## Theorem 2: Log-Capacity Cost Adds Across Independent Feedback Blocks
+
+### Statement
+
+For independent layerwise feedback matrices, let:
+
+```text
+p_l(q_l) = P(Q_l >= q_l)
+C_l(q_l) = -log p_l(q_l)
+```
+
+Then the probability that all alignment thresholds are met is:
+
+```text
+p_all = product_l p_l(q_l)
+```
+
+and total log-volume capacity cost is:
+
+```text
+C_all = -log p_all = sum_l C_l(q_l)
+```
+
+For equal-width hidden layers with width `n` and depth `L`, ordinary FA has:
+
+```text
+D_l approximately n^2
+```
+
+For fixed threshold `q > 0`:
+
+```text
+C_all = Theta(L n^2)
+p_all = exp[-Theta(L n^2)]
+```
+
+For chance-level thresholds:
+
+```text
+q_l = c / D_l
+```
+
+the per-layer cost is constant:
+
+```text
+C_l approximately c/2
+C_all = Theta(L)
+```
+
+### Interpretation
+
+Capacity cost is linear in log-volume units. Raw feasible volume decays
+geometrically.
+
+This resolves the question:
+
+```text
+linear or geometric?
+```
+
+Answer:
+
+```text
+log-capacity cost: linear
+raw volume/probability: geometric/exponential
+```
+
+### Proof Sketch
+
+Independence gives multiplication of feasible probabilities. Taking `-log`
+turns products into sums.
+
+The equal-width MLP scaling follows from:
+
+```text
+D_l = n_l n_{l+1} approximately n^2
+```
+
+and from the high-dimensional capacity approximation.
+
+## Soft-Capacity Null Model
+
+This is not the exact FA theorem, but it is a useful sanity model.
+
+Let:
+
+```text
+P = parameter dimension
+k = dimension of a random constrained subspace
+J = training-set output Jacobian
+K_BP = J J^T
+E = Haar-random k-dimensional subspace
+P_E = projection onto E
+Q = I - P_E
+K_Q = J Q J^T
+```
+
+Then:
+
+```text
+E[P_E] = (k/P) I
+E[Q] = (1 - k/P) I
+E[K_Q | J] = (1 - k/P) K_BP
+```
+
+For residual direction `r`, define:
+
+```text
+e(r) = 1 - (r^T K_Q r) / (r^T K_BP r)
+```
+
+If `v = J^T r`, then:
+
+```text
+e(r) = ||P_E v||^2 / ||v||^2
+```
+
+For Haar-random `E`:
+
+```text
+e(r) ~ Beta(k/2, (P-k)/2)
+E[e(r)] = k/P
+```
+
+This proves the soft-capacity intuition:
+
+```text
+capacity dilutes random constraints through k/P
+but does not make them exactly zero unless k=0 or k/P -> 0
+```
+
+But actual FA is not literally Haar-random deletion of `k` parameter
+dimensions. Experiments showed hard `k` overpredicts real FA gaps. The actual
+FA theorem below is the main result for real FA.
+
+## Theorem 3: Prior-Free Minimax Initialization Bound
+
+### Statement
+
+Let:
+
+```text
+b = normalized feedback direction in R^D
+a = unknown normalized target backward direction in R^D
+```
+
+Let `mu` be any feedback initialization distribution, and define:
+
+```text
+M_mu = E_mu[b b^T]
+trace(M_mu) = 1
+```
+
+Then for target direction `a`:
+
+```text
+E_mu[(a^T b)^2] = a^T M_mu a
+```
+
+So:
+
+```text
+inf_{||a||=1} E_mu[(a^T b)^2]
+  = lambda_min(M_mu)
+  <= 1/D
+```
+
+Therefore:
+
+```text
+sup_mu inf_{||a||=1} E_mu[(a^T b)^2] = 1/D
+```
+
+Isotropic random feedback achieves the bound:
+
+```text
+M_mu = I/D
+```
+
+### Interpretation
+
+Without prior information about the target backward direction, no feedback
+initialization can beat isotropic random feedback in worst-case expected
+squared alignment.
+
+Scale, orthogonality, sparsity, and low rank may affect:
+
+```text
+signal norm
+conditioning
+hardware cost
+rank/channel capacity
+```
+
+but they cannot beat the prior-free angular minimax bound.
+
+### Prior-Aware Corollary
+
+If target directions follow a prior with covariance:
+
+```text
+Sigma_A = E[a a^T]
+```
+
+then:
+
+```text
+E_{a,b}[(a^T b)^2] = trace(Sigma_A M_mu)
+```
+
+With a prior, optimal feedback should allocate mass to high-eigenvalue
+directions of `Sigma_A`. Thus:
+
+```text
+no prior -> isotropic minimax
+with prior -> structure-aware feedback can improve average alignment
+```
+
+## Theorem 4: Actual FA Initial Operator Erosion
+
+### Statement
+
+Fix:
+
+```text
+forward weights W
+training data X,y
+residual r
+ReLU gates induced by W
+```
+
+Assume:
+
+```text
+feedback matrices B_l are independent of W and r
+feedback matrices have zero first moment
+the output layer uses the true output gradient
+```
+
+Let:
+
+```text
+speed_BP = sum_l ||g_l^BP||^2
+speed_FA = sum_l <g_l^BP, g_l^FA>
+e_0(B) = 1 - speed_FA / speed_BP
+```
+
+Then:
+
+```text
+E_B[speed_FA | W,r] = ||g_output^BP||^2
+```
+
+and:
+
+```text
+E_B[e_0 | W,r]
+  = 1 - ||g_output^BP||^2 / sum_l ||g_l^BP||^2
+```
+
+Equivalently:
+
+```text
+E_B[e_0 | W,r] = hidden BP speed share
+```
+
+### Why This Matters
+
+The initial effective FA burden is not the raw number of feedback matrix
+constraints. It is the fraction of BP learning speed carried by hidden-layer
+gradient blocks.
+
+This is a real FA theorem, not a random-subspace proxy.
+
+### Proof Sketch
+
+The output-layer FA gradient equals the BP gradient:
+
+```text
+g_output^FA = g_output^BP
+```
+
+Thus:
+
+```text
+<g_output^BP, g_output^FA> = ||g_output^BP||^2
+```
+
+For any hidden layer, the FA pseudo-gradient contains at least one zero-mean
+independent feedback matrix. Conditional on fixed `W`, `r`, activations, and
+gates:
+
+```text
+E_B[g_l^FA | W,r] = 0
+```
+
+Since `g_l^BP` is fixed under the same conditioning:
+
+```text
+E_B[<g_l^BP, g_l^FA> | W,r]
+  = <g_l^BP, E_B[g_l^FA | W,r]>
+  = 0
+```
+
+Summing layers leaves only the output-layer term.
+
+## Theorem 5: Exact e0 Distribution for One-Hidden-Layer Gaussian FA
+
+### Setup
+
+Consider a one-hidden-layer ReLU MLP:
+
+```text
+x -> W1 -> ReLU -> W2 -> output
+```
+
+with Gaussian feedback:
+
+```text
+B_{a,c} independent Normal(0, sigma_B^2)
+```
+
+For fixed `W1`, `W2`, data, residuals, activations, and gates:
+
+```text
+speed_BP = ||g_output^BP||^2 + ||g_hidden^BP||^2
+speed_FA = ||g_output^BP||^2 + <g_hidden^BP, g_hidden^FA(B)>
+```
+
+The hidden mixed term is linear in `B`:
+
+```text
+<g_hidden^BP, g_hidden^FA(B)> = <C(W,r), B>
+```
+
+where `C(W,r)` is an explicit coefficient matrix determined by data,
+activations, gates, residuals, and the BP hidden gradient.
+
+### Statement
+
+Because `B` is Gaussian:
+
+```text
+<C(W,r), B> ~ Normal(0, sigma_B^2 ||C(W,r)||_F^2)
+```
+
+Therefore:
+
+```text
+e_0(B) = 1 - speed_FA / speed_BP
+```
+
+has exact conditional distribution:
+
+```text
+e_0(B) ~ Normal(mean_e, std_e^2)
+```
+
+with:
+
+```text
+mean_e = 1 - ||g_output^BP||^2 / speed_BP
+std_e  = sigma_B ||C(W,r)||_F / speed_BP
+```
+
+This is the cleanest distributional theorem for actual FA.
+
+### Coefficient Matrix
+
+Let:
+
+```text
+x_n       = input sample n
+delta_n,c = output residual gradient for output c
+gate_n,a  = ReLU gate of hidden unit a
+g_bp_a,d  = BP hidden gradient entry for hidden unit a and input dim d
+```
+
+Then:
+
+```text
+S_{a,c,d} = sum_n gate_n,a x_n,d delta_n,c
+C_{a,c}   = sum_d g_bp_a,d S_{a,c,d}
+```
+
+and:
+
+```text
+<g_hidden^BP, g_hidden^FA(B)> = sum_{a,c} C_{a,c} B_{a,c}
+```
+
+## Experiments
+
+### Experiment 1: Matrix Alignment Beta Law
+
+Object:
+
+```text
+Q = cos^2(A,B)
+```
+
+Theory:
+
+```text
+Q ~ Beta(1/2, (D - 1)/2)
+```
+
+Validation:
+
+```text
+sample many independent A,B
+plot empirical histogram against Beta density
+run QQ/KS checks
+```
+
+Relevant script:
+
+```text
+scripts/static_alignment_beta.py
+```
+
+### Experiment 2: Capacity Scaling
+
+Object:
+
+```text
+C_all = sum_l -log P(Q_l >= q_l)
+```
+
+Theory:
+
+```text
+equal-width MLP, fixed q:
+C_all = Theta(L n^2)
+```
+
+Validation:
+
+```text
+sweep width/depth
+compare computed log-volume capacity with predicted scaling
+```
+
+Relevant scripts:
+
+```text
+scripts/capacity_scaling.py
+scripts/multilayer_capacity_distribution.py
+```
+
+### Experiment 3: Actual FA Initial e0 Mean
+
+Object:
+
+```text
+E_B[e_0 | W,r]
+```
+
+Theory:
+
+```text
+E_B[e_0 | W,r] = hidden BP speed share
+```
+
+Validation:
+
+```text
+fix W and data
+sample many B
+compare empirical mean e0 with predicted hidden BP speed share
+```
+
+Relevant script:
+
+```text
+scripts/actual_fa_initial_operator_moments.py
+```
+
+Representative results:
+
+```text
+width=16 init=0: theory=0.451931 empirical=0.444348
+width=24 init=0: theory=0.312016 empirical=0.312286
+width=32 init=0: theory=0.236058 empirical=0.239714
+width=64 init=0: theory=0.209125 empirical=0.211261
+width=96 init=0: theory=0.259765 empirical=0.259147
+```
+
+### Experiment 4: Exact e0 Distribution
+
+Object:
+
+```text
+e_0(B)
+```
+
+Theory for one-hidden Gaussian FA:
+
+```text
+e_0(B) ~ Normal(mean_e, std_e^2)
+```
+
+Validation:
+
+```text
+fix W and data
+sample 4096 random feedback matrices B per initialization
+plot empirical histogram against predicted Gaussian density
+```
+
+Script:
+
+```text
+scripts/actual_fa_initial_erosion_distribution.py
+```
+
+Run:
+
+```text
+python scripts/actual_fa_initial_erosion_distribution.py \
+  --widths 16 32 64 128 \
+  --init-seeds 3 \
+  --feedback-samples 4096 \
+  --torch-threads 8 \
+  --outdir outputs/actual_fa_initial_erosion_distribution
+```
+
+Main figures:
+
+```text
+outputs/actual_fa_initial_erosion_distribution/e0_distribution_theory_vs_empirical.png
+outputs/actual_fa_initial_erosion_distribution/e0_distribution_moment_calibration.png
+```
+
+Representative results:
+
+| width | init | theory mean | empirical mean | theory std | empirical std | KS | p-value |
+|---:|---:|---:|---:|---:|---:|---:|---:|
+| 16 | 0 | 0.113830 | 0.113413 | 0.027228 | 0.027401 | 0.0150 | 0.314 |
+| 16 | 1 | 0.113705 | 0.113859 | 0.027261 | 0.027278 | 0.0103 | 0.778 |
+| 32 | 0 | 0.121730 | 0.121544 | 0.018952 | 0.018693 | 0.0114 | 0.654 |
+| 32 | 2 | 0.227193 | 0.227691 | 0.031148 | 0.030779 | 0.0166 | 0.206 |
+| 64 | 0 | 0.031155 | 0.031119 | 0.003652 | 0.003583 | 0.0118 | 0.609 |
+| 128 | 0 | 0.023004 | 0.022989 | 0.001630 | 0.001610 | 0.0135 | 0.441 |
+
+Interpretation:
+
+```text
+theory density and empirical random-feedback histogram coincide without fitted
+parameters
+```
+
+This is the current strongest figure.
+
+## Related Work Positioning
+
+### Lillicrap et al.: Random Feedback Can Learn
+
+Lillicrap et al. showed that fixed random feedback weights can support learning
+and observed alignment of forward weights with feedback pathways.
+
+Positioning:
+
+```text
+They show random feedback can be useful.
+We quantify the distributional operator burden random feedback introduces.
+```
+
+Source:
+
+```text
+https://www.nature.com/articles/ncomms13276
+```
+
+### Nøkland: Direct Feedback Alignment
+
+DFA showed random direct feedback can train deep networks in several settings.
+
+Positioning:
+
+```text
+Eventual trainability does not imply BP-equivalent local operator dynamics.
+```
+
+Source:
+
+```text
+https://papers.neurips.cc/paper/6441-direct-feedback-alignment-provides-learning-in-deep-neural-networks
+```
+
+### Refinetti et al.: Align, Then Memorise
+
+Refinetti et al. developed an align-then-memorise picture for DFA dynamics.
+
+Positioning:
+
+```text
+They analyze training-time alignment dynamics.
+Our main theorem isolates the exact initialization-level operator burden and
+its distribution.
+```
+
+Source:
+
+```text
+https://arxiv.org/abs/2011.12428
+```
+
+### Song, Xu, and Lafferty: Overparameterized FA Convergence
+
+Song, Xu, and Lafferty prove convergence of two-layer FA in an
+overparameterized regime and show alignment need not occur without
+regularization.
+
+They decompose:
+
+```text
+K_FA = G + H_FA
+```
+
+where `G` is a positive top-layer kernel and `H_FA` is the hidden
+feedback-dependent term.
+
+Positioning:
+
+```text
+Their result shows FA can converge.
+Our result shows convergence does not imply zero initial operator burden or
+BP-equivalent local learning speed.
+```
+
+Our theorem is consistent with their decomposition:
+
+```text
+E_B[H_FA,0 | W,r] = 0
+E_B[K_FA,0 | W,r] = K_output
+```
+
+Source:
+
+```text
+https://papers.nips.cc/paper/2021/file/a576eafbce762079f7d1f77fca1c5cc2-Paper.pdf
+```
+
+### NTK and Lazy Training
+
+NTK and lazy-training theory justify local tangent-operator dynamics:
+
+```text
+r_{t+1} approximately (I - eta K_t/N) r_t
+```
+
+For BP:
+
+```text
+K_BP = J J^T
+```
+
+For FA:
+
+```text
+K_FA = J J_tilde^T
+```
+
+Positioning:
+
+```text
+We use tangent-operator language to define local FA erosion.
+We do not make finite-time tangent hierarchy the main contribution.
+```
+
+Sources:
+
+```text
+https://papers.nips.cc/paper/8076-neural-tangent-kernel
+https://papers.nips.cc/paper/9063-wide-neural-networks-of-any-depth-evolve-as-linear-models-under-gradient-descent
+```
+
+## Suggested Paper Structure
+
+1. **Introduction**
+   - FA replaces exact backward weights with random feedback.
+   - Existing work asks whether FA can learn.
+   - This paper asks how much distributional operator/capacity burden random
+     feedback introduces.
+
+2. **Preliminaries**
+   - MLP and FA setup.
+   - Matrix alignment statistic.
+   - Local operator erosion statistic.
+
+3. **Distributional Capacity of Random Feedback**
+   - Theorem 1: Beta law.
+   - Log-volume capacity definition.
+   - Theorem 2: multi-layer scaling.
+
+4. **Limits of Prior-Free Initialization**
+   - Theorem 3: minimax bound.
+   - Discussion of isotropic vs structured feedback.
+
+5. **Actual FA Initial Operator Erosion**
+   - Theorem 4: expected erosion equals hidden BP speed share.
+   - Theorem 5: exact Gaussian e0 distribution for one-hidden Gaussian FA.
+   - Relation to Song/Xu/Lafferty `G + H_FA`.
+
+6. **Experiments**
+   - Static alignment Beta distribution.
+   - Capacity scaling.
+   - e0 mean validation.
+   - exact e0 distribution validation.
+
+7. **Discussion**
+   - What is proven vs estimated.
+   - Why hard phase transition was the wrong model.
+   - Future work: time-dependent alignment dynamics and FA tangent hierarchy.
+
+## Suggested Figure List
+
+### Figure 1: Random Matrix Alignment Distribution
+
+Theory:
+
+```text
+Q ~ Beta(1/2, (D-1)/2)
+```
+
+Plot:
+
+```text
+empirical histogram vs Beta density
+```
+
+### Figure 2: Log-Capacity Scaling
+
+Theory:
+
+```text
+C_all = Theta(L n^2)
+```
+
+Plot:
+
+```text
+C_all vs width/depth
+```
+
+### Figure 3: Minimax Initialization Bound
+
+Theory:
+
+```text
+worst-case expected squared alignment <= 1/D
+```
+
+Plot:
+
+```text
+isotropic feedback at bound; non-isotropic improves some directions and worsens
+others
+```
+
+### Figure 4: Actual FA e0 Mean Calibration
+
+Theory:
+
+```text
+E_B[e0 | W,r] = hidden BP speed share
+```
+
+Plot:
+
+```text
+predicted mean vs empirical mean over feedback samples
+```
+
+Existing path:
+
+```text
+outputs/actual_fa_initial_operator_moments/predicted_vs_empirical_initial_erosion_mean.png
+```
+
+### Figure 5: Exact e0 Distribution
+
+Theory:
+
+```text
+e0(B) ~ Normal(mean_e, std_e^2)
+```
+
+Plot:
+
+```text
+theory Gaussian density vs empirical random-feedback histogram
+```
+
+Existing path:
+
+```text
+outputs/actual_fa_initial_erosion_distribution/e0_distribution_theory_vs_empirical.png
+```
+
+This is currently the strongest figure.
+
+## Open Problems and Appendix Material
+
+### Time-Dependent FA Dynamics
+
+At time `t > 0`:
+
+```text
+speed_FA,t = output_speed_t + hidden_alignment_gain_t
+```
+
+where:
+
+```text
+hidden_alignment_gain_t = sum_hidden <g_l^BP(t), g_l^FA(t)>
+```
+
+At initialization, the hidden alignment gain has zero conditional mean. During
+training, `W_t` depends on `B`, so the hidden gain can become positive.
+
+This is actual feedback alignment.
+
+We tested tangent-hierarchy-style estimators:
+
+```text
+K_FA,t = J_t J_tilde_t^T
+K_hat_t = K_0 + t (K_s - K_0)/s
+```
+
+Empirically, early average velocity improves finite-time gap prediction, but
+the infinitesimal derivative `dot_K_0` extrapolated to longer horizons is
+unstable.
+
+Conclusion:
+
+```text
+do not put tangent hierarchy in the main contribution stack
+use it as appendix/future work
+```
+
+Relevant notes:
+
+```text
+notes/30_literature_methods_for_fa_operator_dynamics.md
+notes/31_first_order_tangent_hierarchy_try.md
+```
+
+### Deep Linear or Teacher-Student Full-Time Theory
+
+A controlled finite-time theory may be possible in:
+
+```text
+deep linear FA
+two-layer teacher-student FA/DFA order-parameter dynamics
+```
+
+This is a separate project or appendix-level extension.
+
+## Current Best Abstract Draft
+
+Feedback alignment replaces exact backpropagation weights with fixed random
+feedback matrices, raising a basic quantitative question: how much additional
+capacity or learning-speed burden does random feedback impose relative to BP?
+We formalize this burden distributionally for multilayer perceptrons. First, we
+show that the layerwise squared alignment between a random feedback matrix and
+the corresponding BP backward matrix follows an exact Beta law, yielding a
+log-volume capacity cost whose total multi-layer cost adds across independent
+feedback blocks. Second, we prove a prior-free minimax limit: without
+information about the target backward direction, no feedback initialization can
+exceed the `1/D` worst-case expected squared-alignment bound achieved by
+isotropic random feedback. Third, we derive an actual FA operator result:
+conditional on the forward initialization and residual, the expected initial
+FA learning-speed erosion equals the hidden-layer share of BP gradient energy.
+For one-hidden-layer Gaussian FA, the full conditional erosion distribution is
+Gaussian with computable mean and variance. Large random-feedback ensembles
+validate the predicted alignment and operator-erosion distributions without
+fitted parameters. These results separate eventual trainability from
+BP-equivalent local operator capacity, showing that random feedback may be
+sufficient for learning while still imposing a quantifiable distributional
+operator burden.
+
+## Current Best Title
+
+```text
+Distributional Capacity Bounds for Feedback Alignment in Multilayer Perceptrons
+```
+
+Alternative, more focused:
+
+```text
+Distributional Operator Costs of Feedback Alignment
+```
+
+or:
+
+```text
+Random Feedback Is Not Free: Distributional Operator Bounds for Feedback Alignment
+```
+
+The first title is safer and closer to the original project.