path: root/notes/30_literature_methods_for_fa_operator_dynamics.md

# Literature Review: Methods for FA Operator Dynamics

Question:

```text
Can we move from the exact initial erosion theorem e_0 to a predictive theory
for e_t or finite-time FA/BP gap?
```

This note reviews relevant methods and connects them to our estimator evidence.

## Current State

We now have an actual-FA initialization theorem:

```text
speed_BP = sum_l ||g_l^BP||^2
speed_FA = sum_l <g_l^BP, g_l^FA>

E_B[speed_FA | W,r] = ||g_output^BP||^2

E_B[e_0 | W,r]
  = 1 - ||g_output^BP||^2 / sum_l ||g_l^BP||^2
```

This is not a fit. It follows from zero-mean feedback: hidden FA gradients have
zero conditional mean at initialization, while the output-layer FA gradient is
exactly the BP gradient.

The unresolved part is:

```text
what happens after W_t becomes dependent on B?
```

At later time:

```text
speed_FA,t = output_speed_t + hidden_alignment_gain_t
```

where:

```text
hidden_alignment_gain_t = sum_hidden <g_l^BP(t), g_l^FA(t)>
```

At `t=0`, this hidden gain has zero conditional mean. During training it can
become positive because the forward weights become statistically coupled to the
fixed feedback matrices.

## Literature Takeaways

### 1. NTK and frozen-kernel dynamics

Jacot et al. introduced the Neural Tangent Kernel and showed that, in the
infinite-width limit, the function follows kernel gradient dynamics and the NTK
stays constant during training.

Source:

```text
https://papers.nips.cc/paper/8076-neural-tangent-kernel
```

Useful for us:

```text
r_{t+1} ≈ (I - eta K_t / N) r_t
```

This justifies our tangent-operator estimator. For BP:

```text
K_BP = J J^T
```

For FA:

```text
K_FA = J J_tilde^T
```

But NTK alone only gives a clean frozen-kernel theorem when the kernel is
approximately constant.

### 2. Wide networks as linear models / lazy training

Lee et al. show that wide networks of any depth are governed, in the infinite
width limit, by the first-order Taylor expansion around initialization. They
also report strong finite-width empirical agreement.

Source:

```text
https://papers.nips.cc/paper/9063-wide-neural-networks-of-any-depth-evolve-as-linear-models-under-gradient-descent
```

Chizat, Oyallon, and Bach frame this as lazy training and provide path-distance
bounds between nonlinear and linearized optimization.

Source:

```text
https://papers.neurips.cc/paper_files/paper/2019/hash/ae614c557843b1df326cb29c57225459-Abstract.html
```

Useful for us:

```text
fixed K(0) is a local theorem, not a long-horizon theorem
```

This matches our estimator evidence:

```text
T=1 and T=5: fixed K(0) predicts FA/BP gap almost exactly
T=50 and beyond: fixed K(0) drifts out of regime
```

So the correct paper language is:

```text
local gap distribution: fixed initial FA tangent operator
finite-time gap distribution: time-varying FA tangent operator
```

### 3. Neural Tangent Hierarchy for kernel evolution

Huang and Yau derive a Neural Tangent Hierarchy (NTH), an infinite hierarchy of
ODEs for the evolution of finite-width NTKs. Truncating the hierarchy gives
controlled approximations to NTK dynamics under width/data conditions.

Source:

```text
https://proceedings.mlr.press/v119/huang20l.html
```

This is the closest existing method for our `t > 0` problem.

For us, the analogous object is not just NTK:

```text
K_FA,t = J_t J_tilde_t^T
```

Its derivative has the same structure:

```text
dK_FA,t/dt
  = dJ_t/dt J_tilde_t^T + J_t dJ_tilde_t^T/dt
```

and both terms depend on higher-order tangent objects. This suggests a direct
method:

```text
derive an FA tangent hierarchy
truncate at first order:
K_FA,t ≈ K_FA,0 + t dot_K_FA,0
```

This is exactly what our best estimator is doing empirically when it uses early
operator velocity:

```text
K_hat_t = K_0 + t (K_s - K_0) / s
```

The difference is that the estimator measures the velocity, while a theorem
would compute or bound it.

### 4. Song, Xu, and Lafferty: perturbative FA kernel decomposition

Song, Xu, and Lafferty analyze two-layer FA and prove overparameterized
convergence. Their key structural decomposition is:

```text
K_FA = G + H_FA
```

where `G` is the positive top-layer kernel and `H_FA` is the hidden
feedback-dependent term. They explicitly treat `H_FA` as non-PSD and
perturbative in the wide regime.

Source:

```text
https://papers.nips.cc/paper/2021/file/a576eafbce762079f7d1f77fca1c5cc2-Paper.pdf
```

Useful for us:

Our initial theorem is the residual-direction version of their decomposition:

```text
E_B[H_FA,0 | W,r] = 0
```

Their proof style suggests a way to bound finite-time FA cost:

```text
speed_FA,t = r_t^T G_t r_t + r_t^T H_FA,t r_t
```

Then:

```text
operator erosion = 1 - speed_FA,t / speed_BP,t
```

If `H_FA,t` remains perturbative, FA can still converge because `G_t` is
positive, but it need not be BP-equivalent. This gives a rigorous way to say:

```text
convergence does not imply zero BP/FA operator gap
```

without contradicting their theorem.

### 5. Refinetti et al.: align-then-memorise order parameters

Refinetti et al. derive analytical dynamics for DFA in shallow nonlinear
teacher-student settings and identify an alignment phase followed by a
memorisation phase. They also analyze deep linear alignment matrices and
conditioning.

Source:

```text
https://arxiv.org/abs/2011.12428
```

Useful for us:

They show the right finite-time state variables are low-dimensional alignment
order parameters, not raw parameter counts. Their theory predicts learning
curves by ODEs over overlaps.

This suggests an architecture-level route:

```text
derive ODEs for:
  residual norm
  output BP speed share
  hidden alignment gain
  layerwise B-W overlaps
```

Then:

```text
e_t = 1 - (output_speed_t + hidden_alignment_gain_t) / speed_BP,t
```

This would be more faithful than hard-k capacity exhaustion.

### 6. Deep linear exact dynamics as a solvable testbed

Saxe, McClelland, and Ganguli solve learning dynamics in deep linear networks
and show rich nonlinear learning phenomena despite linear input-output maps.

Source:

```text
https://arxiv.org/abs/1312.6120
```

Useful for us:

Deep linear FA may be the cleanest place to get a full time-dependent theorem:

```text
closed dynamics for W_t and B-induced alignment
closed e_t or gap_T formula in singular modes
```

This would not replace nonlinear MLP experiments, but it could give a rigorous
finite-time bridge.

## What Our Estimator Experience Says

Our experiments already rule out one bad path:

```text
hard-k random-subspace model does not predict real FA gap
```

It overpredicts positive-margin gaps and treats FA as if it deletes parameter
directions. Real FA instead keeps an output-layer kernel and gradually learns
hidden alignment.

Our experiments support this path:

```text
initial actual FA moment theorem
  -> e_0 is hidden BP speed share

early operator velocity
  -> predicts finite-time gap much better than fixed K(0)

time-varying operator product
  -> correct object for larger T
```

Empirical signs:

```text
fixed K(0), small T: accurate
fixed K(0), larger T: BP estimated too strong, FA estimated too weak
```

Interpretation:

```text
BP: finite-width feature drift makes initial BP kernel too optimistic
FA: alignment gain makes initial FA operator too pessimistic
```

Therefore the theory should model:

```text
K_BP,t drift
K_FA,t alignment gain
```

not just initial random geometry.

## Candidate Methods

### Method A: FA tangent hierarchy

Define:

```text
K_FA,t = J_t J_tilde_t^T
S_FA,t = (K_FA,t + K_FA,t^T) / 2
e_t = 1 - (r_t^T S_FA,t r_t) / (r_t^T K_BP,t r_t)
```

Derive:

```text
dK_FA,t/dt
  = dJ_t/dt J_tilde_t^T + J_t dJ_tilde_t^T/dt
```

Both terms are higher-order tangent objects, exactly like NTH.

Short-time theorem target:

```text
K_FA,t = K_FA,0 + t dot_K_FA,0 + error
```

and:

```text
gap_T = fixed-K gap + first-order drift correction + bounded remainder
```

This is the most principled extension of our estimator.

Pros:

```text
closest to our current successful estimator
no fitted scalar
general MLP-compatible
can become a theorem with smoothness/lazy assumptions
```

Cons:

```text
heavy notation
dot_K_FA,0 is a third-order object
full distribution over B may require Wick/CLT approximations
```

### Method B: Song-style `G + H` perturbation

Decompose:

```text
K_FA,t = K_output,t + H_FA,t
```

where:

```text
K_output,t = output-layer BP kernel
H_FA,t = hidden feedback-dependent contribution
```

At initialization:

```text
E_B[r_0^T H_FA,0 r_0 | W,r] = 0
```

At time `t`, alignment appears through:

```text
r_t^T H_FA,t r_t
```

The theorem target:

```text
finite-time FA/BP erosion
  = hidden BP speed share
    - hidden alignment gain / speed_BP
```

Then bound or approximate the hidden alignment gain.

Pros:

```text
simple conceptual language
directly compatible with Song/Xu/Lafferty
explains convergence without BP equivalence
```

Cons:

```text
may give bounds more easily than exact distribution
deep nonlinear case still hard
```

### Method C: order-parameter ODEs

Follow Refinetti-style teacher-student analysis.

Pick a controlled task:

```text
Gaussian inputs
teacher network labels
two-layer or deep linear student
large input dimension
finite hidden width
```

Track overlaps:

```text
student-teacher overlaps
student-student overlaps
feedback-output overlaps
layerwise alignment matrices
```

Then compute:

```text
e_t and gap_T from ODE trajectories
```

Pros:

```text
architecture-level prediction
can predict full learning curves
natural distribution over B and initialization
```

Cons:

```text
narrower task setting
harder to match our random-label memorization experiments
less general than tangent-operator estimator
```

### Method D: deep-linear exact FA dynamics

Use linear networks as the tractable theorem setting.

For deep linear networks:

```text
f(x) = W_L ... W_1 x
```

FA replaces backward products by fixed random feedback products. Because gates
are absent, the dynamics can be written as matrix ODEs.

Target:

```text
derive exact or perturbative e_t
show e_0 is output-speed share
show alignment gain grows from B-W coupling
compare BP/FA finite-time loss in singular modes
```

Pros:

```text
rigorous and readable
connects to Saxe-style dynamics and Refinetti deep-linear alignment
```

Cons:

```text
not the final nonlinear MLP setting
may be viewed as a sanity theorem unless paired with MLP experiments
```

## Recommended Plan

Use a three-level theory stack.

### Level 1: exact local theorem

Keep the current exact theorem:

```text
E_B[e_0 | W,r] = hidden BP speed share
```

Add conditional variance/distribution for the simplest case:

```text
one-hidden-layer Gaussian feedback:
speed_FA = output_speed + Gaussian hidden term
```

This gives a theory-predicted distribution graph, not just a mean.

### Level 2: short-time FA tangent hierarchy

Define:

```text
dot_K_FA,0 = derivative of J J_tilde^T under the FA update
```

Then prove a short-time expansion:

```text
K_FA,t ≈ K_FA,0 + t dot_K_FA,0
```

and propagate residuals with the corresponding product. This formalizes our
successful early-operator estimator.

If computing `dot_K_FA,0` exactly is too heavy, use `K_s - K_0` as an estimator
and label it correctly:

```text
conditional tangent-hierarchy estimator
```

### Level 3: controlled full-time theory

For a theorem-only full-time result, use either:

```text
deep linear FA
```

or:

```text
two-layer teacher-student FA/DFA order-parameter ODE
```

This can show explicitly how hidden alignment gain grows and why final
convergence does not imply zero finite-time BP/FA gap.

## What Not To Do

Do not return to:

```text
hard margin predicts exact transition
```

The dense T=30000 experiment showed a soft ramp, not a hard phase transition.

Do not introduce:

```text
post-hoc scalar correction
```

The paper should use either:

```text
exact distribution / exact moment
```

or:

```text
conditional estimator with observed early operator information
```

Do not claim:

```text
architecture alone predicts all finite-time FA/BP gaps
```

unless we derive an ODE or hierarchy closure that predicts `K_FA,t`.

## Best Current Claim

The cleanest claim after the literature review is:

```text
Random feedback has an exact, nonzero initial operator erosion equal to the
hidden BP speed share. At finite time, feedback alignment is precisely the
growth of a hidden alignment-gain term. Existing NTK hierarchy and
align-then-memorise theory suggest two principled ways to model this gain:
short-time tangent-operator hierarchy or low-dimensional order-parameter ODEs.
Our current time-varying operator estimator is the empirical version of the
first route and should be presented as such, not as a fitted curve.
```

This makes the theory stack honest:

```text
e_0 theorem
short-time tangent hierarchy / estimator
controlled ODE or deep-linear full-time theory
large-scale MLP trajectory validation
```