Revise capacity theory to soft erosion

1 files changed, 35 insertions, 12 deletions

diff --git a/notes/18_contribution_roadmap.md b/notes/18_contribution_roadmap.md
index 179bd5d..4805a19 100644
--- a/notes/18_contribution_roadmap.md
+++ b/notes/18_contribution_roadmap.md
@@ -33,7 +33,7 @@ C_l(q_l) = -log P(Q_l >= q_l)
 This formalizes what it means for feedback alignment to consume parameter
 direction volume.
 
-## 2. Scaling Law and Redundancy Exhaustion
+## 2. Scaling Law and Soft Capacity Erosion
 
 Across independent layers, raw feasible volume multiplies while log-capacity
 cost adds:
@@ -50,17 +50,37 @@ C_all = Θ(L n^2)
 p_all = exp[-Θ(L n^2)]
 ```
 
-Functional loss does not need to appear immediately. If total parameter
-dimension is `P`, task dimension is `d`, and alignment imposes `k` generic
-constraints, then hard functional rank loss is:
+Functional loss should be modeled as soft erosion, not a hard step. If total
+parameter dimension is `P`, task-sensitive dimension is `d`, and alignment
+imposes a generic `k`-dimensional constraint subspace, then its expected overlap
+with the task subspace is:
+
+```text
+E[tr(P_E P_S)] = k d / P
+```
+
+Equivalently, if `J` is the training-set output Jacobian and a random
+`k`-dimensional constraint leaves projection `Q`, then:
+
+```text
+K_k = J Q J^T
+E[K_k | J] = (1 - k/P) K_BP
+```
+
+So the FA/BP gap need not wait until redundant directions are fully exhausted.
+Redundancy suppresses the expected task-relevant alignment burden, but does not
+make it exactly zero.
+
+Hard functional rank loss remains a limiting sanity check:
 
 ```text
 Δd_hard = max(0, k - (P - d))
 ```
 
-So the FA/BP gap should open when redundant directions are exhausted.
+This threshold describes exact rank collapse, not finite-time loss,
+conditioning loss, or effective-dimension loss.
 
-This is the phase-transition contribution.
+This is the soft capacity-erosion contribution.
 
 ## 3. Prior-Free Minimax Initialization Bound
 
@@ -136,8 +156,8 @@ Experiments should validate three levels:
    ```text
    Q_l ~ Beta(1/2, (D_l - 1)/2)
    ```
-2. capacity and redundancy transition:
-   FA/BP gap opens near the hard FA capacity margin crossing;
+2. capacity and soft erosion:
+   FA/BP gap distribution changes smoothly with hard FA capacity margin;
 3. tangent-operator trajectory distributions:
    predicted gap distributions overlap empirical trajectory distributions.
 
@@ -167,16 +187,19 @@ This separation avoids overclaiming while still giving a coherent theory chain.
 
 ## Next Priority
 
-The weakest current visual is the phase-transition contribution.
+The weakest current visual was the phase-transition contribution; the dense
+long-training sweep now shows that the correct phenomenon is a soft ramp, not a
+hard kink.
 
 The desired figure should show:
 
 ```text
-capacity margin decreases -> redundant directions exhausted -> FA/BP gap opens
-and then grows
+capacity margin decreases -> task-relevant alignment overlap increases ->
+FA/BP gap distribution grows smoothly
 ```
 
 The plot should use hard FA capacity margin on the x-axis, place the zero
-margin as a vertical reference, and show BP/FA train loss or FA-BP train gap.
+margin as a vertical reference, and show the FA-BP train-gap distribution on a
+log scale.
 The most direct target is a width sweep on a random-label task, because random
 labels make the task dimension controllable and force memorization capacity.