# First-Order FA Tangent-Hierarchy Try

This note records the first direct attempt to use tangent-hierarchy ideas for
finite-time FA/BP gap prediction.

## Goal

Test whether finite-time gap can be improved by replacing the fixed initial
operator:

```text
K_t = K_0
```

with a first-order operator trajectory:

```text
K_t = K_0 + t V_0
```

where `V_0` is either:

```text
early average velocity: (K_s - K_0) / s
```

or:

```text
infinitesimal derivative: (K_epsilon - K_0) / epsilon
```

This is the smallest test of the FA tangent-hierarchy idea.

## Experiment 1: Early Average Velocity

Script:

```text
scripts/compressed_operator_predictor.py
```

Run:

```text
python scripts/compressed_operator_predictor.py \
  --width 64 \
  --hidden-layers 2 \
  --train-samples 128 \
  --target-steps 50 \
  --early-steps 1 2 5 \
  --init-seeds 3 \
  --feedback-seeds 6 \
  --lr 1e-3 \
  --torch-threads 8 \
  --outdir outputs/fa_tangent_hierarchy_first_order_try
```

Plots:

```text
outputs/fa_tangent_hierarchy_first_order_try/first_order_prediction_scatter_by_early_step.png
outputs/fa_tangent_hierarchy_first_order_try/first_order_error_metrics_by_early_step.png
outputs/fa_tangent_hierarchy_first_order_try/first_order_gap_distribution_overlay.png
```

Summary:

| early s | predictor | MAE | bias | corr |
|---:|---|---:|---:|---:|
| 1 | fixed K0 | 0.023790 | +0.023790 | 0.941852 |
| 1 | linear velocity | 0.022925 | -0.020753 | 0.742632 |
| 2 | fixed K0 | 0.023790 | +0.023790 | 0.941852 |
| 2 | linear velocity | 0.012290 | -0.008418 | 0.815872 |
| 5 | fixed K0 | 0.023790 | +0.023790 | 0.941852 |
| 5 | linear velocity | 0.012232 | -0.012232 | 0.960370 |
| 5 | retangent | 0.016770 | +0.016770 | 0.971053 |

Interpretation:

```text
early average velocity improves magnitude substantially
```

At `s=2` or `s=5`, MAE is nearly cut in half relative to fixed `K0`.

But:

```text
linear extrapolation shifts from positive bias to negative bias
```

So the first-order average velocity captures the direction of operator drift,
but extrapolating it linearly to `T=50` overcorrects.

This is useful: the tangent-hierarchy route is real, but the model should not
be a naive global linear extrapolation.

## Experiment 2: Infinitesimal Derivative Probe

Script:

```text
scripts/fa_tangent_hierarchy_derivative_probe.py
```

Run:

```text
python scripts/fa_tangent_hierarchy_derivative_probe.py \
  --width 64 \
  --hidden-layers 2 \
  --train-samples 128 \
  --target-steps 50 \
  --epsilon 0.05 \
  --init-seeds 3 \
  --feedback-seeds 6 \
  --lr 1e-3 \
  --torch-threads 8 \
  --outdir outputs/fa_tangent_hierarchy_derivative_probe_eps005
```

Result:

```text
fixed MAE      = 0.0237897
fixed bias     = +0.0237897
derivative MAE = 270.362
derivative bias= -270.212
```

The infinitesimal first derivative is unusable when extrapolated linearly to
`T=50`.

Likely reasons:

```text
1. K_t is not globally linear for 50 steps;
2. the first derivative creates an indefinite operator path that can become
   unstable when extrapolated;
3. ReLU gates and FA alignment dynamics make the actual early average velocity
   very different from the infinitesimal derivative;
4. higher-order hierarchy terms are not small over this horizon.
```

This is not a normalization bug. The same setup shows fixed `K0` is good at
very short horizons and that early average velocity improves finite-time
magnitude.

## Consequence

The simple theorem:

```text
K_t = K_0 + t dot_K_0
```

is not enough for practical `T=50` prediction.

The better finite-time object is:

```text
K_t = K_0 + A_t
```

where `A_t` should be a bounded short-time alignment-gain path, not an
unbounded linear extrapolation.

Empirically, the useful object is:

```text
average early velocity over s=2 or s=5
```

not the infinitesimal `dot_K_0`.

## Revised Theory Direction

We should pursue one of these:

1. **Integrated short-time hierarchy**

   Predict:

   ```text
   K_s - K_0 = integral_0^s dot_K_u du
   ```

   rather than only `dot_K_0`.

2. **Stable alignment-gain scalar**

   Decompose:

   ```text
   speed_FA,t = output_speed_t + hidden_alignment_gain_t
   ```

   and model the hidden alignment gain as a bounded increasing process.

3. **Piecewise retangent estimator**

   For empirical validation, use:

   ```text
   measured K_s
   ```

   as a conditional estimator, and do not claim it is architecture-only.

The most paper-safe claim:

```text
The exact e_0 theorem gives the initial operator cost. For finite time, the
successful predictor is an early-time integrated tangent-operator estimator,
which is an empirical finite-difference version of a truncated FA tangent
hierarchy. A pure infinitesimal first-derivative extrapolation is not stable
over nonlocal horizons.
```

This result is useful because it prevents us from overclaiming a too-simple
first-order theory.