summaryrefslogtreecommitdiff
path: root/notes/07_minimal_transition_theory.md
diff options
context:
space:
mode:
Diffstat (limited to 'notes/07_minimal_transition_theory.md')
-rw-r--r--notes/07_minimal_transition_theory.md104
1 files changed, 104 insertions, 0 deletions
diff --git a/notes/07_minimal_transition_theory.md b/notes/07_minimal_transition_theory.md
new file mode 100644
index 0000000..ced7e61
--- /dev/null
+++ b/notes/07_minimal_transition_theory.md
@@ -0,0 +1,104 @@
+# Minimal Transition Theory
+
+The cleanest correction is not to add a separate FA burden coefficient first.
+Start by calibrating the task dimension from BP alone.
+
+## Problem
+
+The hard theory used:
+
+\[
+d_{\mathrm{task}}=Nn_L=512.
+\]
+
+With this value, the smooth theory peak is near:
+
+\[
+M_{\mathrm{FA}}\approx -273,
+\]
+
+while 256 empirical FA trajectories peak at:
+
+\[
+M_{\mathrm{FA}}=-318.
+\]
+
+## BP-Only Calibration
+
+Fit the BP train curve with:
+
+\[
+L_{\mathrm{BP}}(P)
+\approx
+s\cdot \operatorname{softplus}_{\tau}(d_{\mathrm{task}}^{\mathrm{eff}}-P).
+\]
+
+The BP-only fit gives:
+
+\[
+d_{\mathrm{task}}^{\mathrm{eff}}\approx 437,
+\qquad
+\tau\approx 50.
+\]
+
+So BP is stronger than the naive parameter count suggests:
+
+\[
+d_{\mathrm{task}}^{\mathrm{eff}}
+\approx
+0.85\,Nn_L.
+\]
+
+## Minimal FA Prediction
+
+Use the same hard FA burden:
+
+\[
+K_{\mathrm{FA}}=\sum_l(D_l-1),
+\]
+
+but replace the task dimension:
+
+\[
+M_{\mathrm{FA}}^{\mathrm{min}}
+=
+P-K_{\mathrm{FA}}-d_{\mathrm{task}}^{\mathrm{eff}}.
+\]
+
+Equivalently, when plotted on the original hard-margin axis
+
+\[
+P-K_{\mathrm{FA}}-Nn_L,
+\]
+
+the theory shifts left by:
+
+\[
+Nn_L-d_{\mathrm{task}}^{\mathrm{eff}}
+\approx
+75.
+\]
+
+This explains the observed left shift without introducing a fitted FA burden
+coefficient.
+
+Generated figure:
+
+`outputs/downstream_capacity_random_main_fast/capacity_transition_bp_calibrated_p2_256.png`
+
+## Recommendation
+
+Use this as the main downstream theory:
+
+1. Matrix/capacity theory remains exact and uses the hard/log-volume definitions.
+2. Downstream performance uses a BP-calibrated effective task dimension.
+3. FA capacity loss is then tested with no extra FA-specific fitting parameter.
+
+Only add an effective FA burden coefficient
+
+\[
+K_{\mathrm{FA}}^{\mathrm{eff}}=\alpha K_{\mathrm{FA}}
+\]
+
+as an appendix ablation if needed. It improves fit but makes the theory look
+more ad hoc.