# Soft-Erosion Theory Distribution Validation Attempt

We tested whether the revised soft-erosion theory can predict the empirical
FA/BP train-gap distribution from the dense `T=30000` sweep.

## Data

Empirical run:

```text
outputs/phase_transition_dense_T30000_352traj
```

Validation output:

```text
outputs/soft_erosion_theory_vs_empirical
```

Main plots:

```text
outputs/soft_erosion_theory_vs_empirical/gap_distribution_theory_vs_empirical.png
outputs/soft_erosion_theory_vs_empirical/local_erosion_distribution_theory_vs_empirical.png
outputs/soft_erosion_theory_vs_empirical/local_operator_scalar_gap_theory_vs_empirical.png
```

Script:

```text
scripts/soft_erosion_theory_vs_empirical.py
```

## Theory 1: Hard-k Random-Subspace Gap Distribution

This test uses the null model:

```text
P = parameter count
k = hard FA constraint count
rho ~ Beta((P-k)/2, k/2)
K_FA ≈ rho K_BP
```

Then it predicts finite-time loss from the BP initial tangent spectrum:

```text
r_T_BP = exp(-eta T K_BP / N) r_0
r_T_FA = exp(-eta T rho K_BP / N) r_0
gap_T = (||r_T_FA||^2 - ||r_T_BP||^2) / (2N)
```

This is not fitted. The only random quantity is the random-subspace theory
distribution for `rho`.

Result:

```text
gap_distribution_theory_vs_empirical.png
```

The hard-k random-subspace theory strongly overpredicts the empirical gap,
especially on the positive-margin side.

Examples:

| width | margin | hard-k theory mean | empirical mean |
|---:|---:|---:|---:|
| 40 | 130 | 0.040374 | 0.000162 |
| 36 | 66 | 0.050133 | 0.000791 |
| 32 | 2 | 0.064534 | 0.003729 |
| 28 | -62 | 0.103630 | 0.014195 |
| 24 | -126 | 0.121998 | 0.046464 |
| 20 | -190 | 0.151140 | 0.146803 |

So the hard constraint count `k` is not the effective burden. It is too
pessimistic.

## Theory 2: Local One-Step Erosion

We measured the actual initial local operator erosion for each feedback seed:

```text
speed_ratio = <g_BP, g_FA> / <g_BP, g_BP>
erosion = 1 - speed_ratio
```

This directly estimates:

```text
1 - (r^T S_FA r) / (r^T K_BP r)
```

where:

```text
S_FA = (K_FA + K_FA^T) / 2
```

The comparison:

```text
local_erosion_distribution_theory_vs_empirical.png
```

shows that real FA local erosion is much smaller than the hard-k Beta null.

Examples:

| width | empirical local erosion mean | hard-k Beta erosion mean |
|---:|---:|---:|
| 24 | 0.3907 | about 0.634 |
| 32 | 0.2338 | about 0.691 |
| 40 | 0.3381 | about 0.733 |

Again, hard `k` overcounts effective burden.

## Theory 3: Local-Operator Scalar Gap Prediction

We then used the measured initial speed ratio as a no-fit scalar operator
prediction:

```text
K_FA ≈ speed_ratio(B) K_BP
```

and propagated the BP tangent spectrum for `T=30000`.

Result:

```text
local_operator_scalar_gap_theory_vs_empirical.png
```

This improves over hard-k random subspace but is still not sufficient.

Examples:

| width | margin | local-operator theory mean | empirical mean |
|---:|---:|---:|---:|
| 40 | 130 | 0.002701 | 0.000162 |
| 36 | 66 | 0.004612 | 0.000791 |
| 32 | 2 | 0.005276 | 0.003729 |
| 28 | -62 | 0.026043 | 0.014195 |
| 24 | -126 | 0.047863 | 0.046464 |
| 22 | -158 | 0.037249 | 0.100526 |
| 20 | -190 | 0.071783 | 0.146803 |

It matches around `width=24` and is roughly right near `width=32`, but:

1. it overpredicts the positive-margin tail;
2. it underpredicts the more negative-margin regime;
3. it is not monotone enough because a single initial scalar cannot capture
   training-time alignment and feature drift.

## Conclusion

The current result is not a successful validation of the hard-k random-subspace
theory.

It gives a sharper diagnosis:

```text
hard FA constraint count k is not k_eff.
```

The empirical gap is controlled by an effective operator erosion that evolves
during training:

```text
K_FA,t = J_t J_tilde_t^T
```

not by a fixed random deletion of a `k`-dimensional subspace.

## Revised Theory Target

The next theory should predict a distribution over:

```text
e_t(B) = 1 - (r_t^T S_FA,t r_t) / (r_t^T K_BP,t r_t)
```

or its cumulative finite-time form:

```text
r_T_FA = product_t (I - eta K_FA,t / N) r_0
r_T_BP = product_t (I - eta K_BP,t / N) r_0
```

The right effective capacity is therefore:

```text
k_eff(B,t) = P * e_t(B)
```

or a spectral version of this quantity, not the raw hard matrix constraint
count.

## Relation to Song, Xu, and Lafferty

`Convergence and Alignment of Gradient Descent with Random Backpropagation
Weights` is highly relevant and must be read carefully next.

Current hypothesis:

```text
Their overparameterized convergence theorem is compatible with our findings.
It shows FA can converge; it does not imply BP-equivalent finite-time tangent
capacity.
```

We need to check whether their alignment dynamics imply a decay law for
`e_t(B)` or `k_eff(B,t)`.