# Estimator vs Nonzero-Cost Theorem

We do not need the estimator to be the main theory.

The estimator answers:

```text
given an initialization, feedback matrix, data, and early operator state,
how large is the finite-time FA/BP train-gap?
```

The theorem we want for the paper can be weaker and cleaner:

```text
redundant parameters do not make random feedback alignment cost-free.
```

These are different claims.

## Avoid the Wrong Claim

Do not claim:

```text
overparameterized FA cannot reach zero training error.
```

That is too strong and likely false in important settings. Existing
overparameterized FA results prove convergence to zero training error in
specific two-layer settings.

The right claim is:

```text
overparameterization does not imply zero FA/BP operator gap or zero alignment
cost.
```

FA may eventually fit the task while still having a nonzero finite-time tangent
capacity loss relative to BP.

## Clean Random-Subspace Theorem

Let:

```text
P = parameter dimension
k = effective alignment burden dimension
J = training-set output Jacobian
K_BP = J J^T
r = current residual
E = random k-dimensional constrained subspace, independent of J and r
P_E = projection onto E
Q = I - P_E
K_Q = J Q J^T
```

Define the one-step operator erosion:

```text
e(r) = 1 - (r^T K_Q r) / (r^T K_BP r)
```

Assume:

```text
r^T K_BP r > 0
```

Then by rotational invariance:

```text
E[P_E] = (k/P) I
E[Q] = (1 - k/P) I
```

Therefore:

```text
E[e(r) | J, r] = k/P
```

Moreover, if:

```text
v = J^T r
```

then:

```text
(r^T K_Q r) / (r^T K_BP r)
= ||Q v||^2 / ||v||^2
```

and for Haar-random `E`:

```text
||P_E v||^2 / ||v||^2 ~ Beta(k/2, (P-k)/2)
```

So:

```text
e(r) ~ Beta(k/2, (P-k)/2)
```

under the random-subspace model.

This proves:

```text
P(e(r) = 0) = 0
```

for `k > 0`, and:

```text
E[e(r)] = k/P
```

Thus redundant capacity reduces the expected erosion only through `k/P`; it
does not make the erosion exactly zero unless:

```text
k = 0
```

or in a scaling limit:

```text
k/P -> 0
```

## Scaling Consequence

If:

```text
k is fixed and P -> infinity
```

then:

```text
E[e(r)] -> 0
```

so the alignment burden vanishes asymptotically.

If:

```text
k/P -> alpha > 0
```

then:

```text
E[e(r)] -> alpha
```

and the relative operator erosion persists.

This is the theorem that directly supports the sell:

```text
capacity can dilute random feedback alignment burden, but it cannot make it
free under proportional scaling.
```

## Role of the Estimator

The estimator is not needed to prove nonzero cost.

It is needed for:

1. checking whether the random-subspace model is too pessimistic;
2. measuring `k_eff` for real FA rather than hard `k`;
3. predicting finite-time train-gap distributions after training dynamics,
   alignment, and feature drift.

So the paper can separate:

```text
Theorem: random alignment burden has nonzero expected operator erosion.
Estimator: finite-time dynamics convert operator erosion into observed train-gap.
```

This separation prevents overclaiming.

## Revised Contribution Logic

Contribution 2 should not promise exact gap prediction.

It should prove:

```text
no free redundant capacity:
E[e(r)] = k_eff/P, hence nonzero when k_eff/P is nonzero.
```

Contribution 4 can then say:

```text
to quantify actual finite-time gap, use tangent-operator estimators.
```

This makes the theorem clean and the estimator useful without forcing one to do
the other's job.