summaryrefslogtreecommitdiff
path: root/notes/07_minimal_transition_theory.md
diff options
context:
space:
mode:
authorYurenHao0426 <Blackhao0426@gmail.com>2026-06-02 15:13:04 -0500
committerYurenHao0426 <Blackhao0426@gmail.com>2026-06-02 15:13:04 -0500
commitdaf0a6dc795235864d8279908320801f3c7b66b5 (patch)
treeebe59a8768889cf5430c6a4ec142a2e3a7693a1b /notes/07_minimal_transition_theory.md
parentb436e6c033c2c3b37d2ee26efe0dbe28381117dc (diff)
Replace transition theory with tangent capacity
Diffstat (limited to 'notes/07_minimal_transition_theory.md')
-rw-r--r--notes/07_minimal_transition_theory.md104
1 files changed, 0 insertions, 104 deletions
diff --git a/notes/07_minimal_transition_theory.md b/notes/07_minimal_transition_theory.md
deleted file mode 100644
index ced7e61..0000000
--- a/notes/07_minimal_transition_theory.md
+++ /dev/null
@@ -1,104 +0,0 @@
-# Minimal Transition Theory
-
-The cleanest correction is not to add a separate FA burden coefficient first.
-Start by calibrating the task dimension from BP alone.
-
-## Problem
-
-The hard theory used:
-
-\[
-d_{\mathrm{task}}=Nn_L=512.
-\]
-
-With this value, the smooth theory peak is near:
-
-\[
-M_{\mathrm{FA}}\approx -273,
-\]
-
-while 256 empirical FA trajectories peak at:
-
-\[
-M_{\mathrm{FA}}=-318.
-\]
-
-## BP-Only Calibration
-
-Fit the BP train curve with:
-
-\[
-L_{\mathrm{BP}}(P)
-\approx
-s\cdot \operatorname{softplus}_{\tau}(d_{\mathrm{task}}^{\mathrm{eff}}-P).
-\]
-
-The BP-only fit gives:
-
-\[
-d_{\mathrm{task}}^{\mathrm{eff}}\approx 437,
-\qquad
-\tau\approx 50.
-\]
-
-So BP is stronger than the naive parameter count suggests:
-
-\[
-d_{\mathrm{task}}^{\mathrm{eff}}
-\approx
-0.85\,Nn_L.
-\]
-
-## Minimal FA Prediction
-
-Use the same hard FA burden:
-
-\[
-K_{\mathrm{FA}}=\sum_l(D_l-1),
-\]
-
-but replace the task dimension:
-
-\[
-M_{\mathrm{FA}}^{\mathrm{min}}
-=
-P-K_{\mathrm{FA}}-d_{\mathrm{task}}^{\mathrm{eff}}.
-\]
-
-Equivalently, when plotted on the original hard-margin axis
-
-\[
-P-K_{\mathrm{FA}}-Nn_L,
-\]
-
-the theory shifts left by:
-
-\[
-Nn_L-d_{\mathrm{task}}^{\mathrm{eff}}
-\approx
-75.
-\]
-
-This explains the observed left shift without introducing a fitted FA burden
-coefficient.
-
-Generated figure:
-
-`outputs/downstream_capacity_random_main_fast/capacity_transition_bp_calibrated_p2_256.png`
-
-## Recommendation
-
-Use this as the main downstream theory:
-
-1. Matrix/capacity theory remains exact and uses the hard/log-volume definitions.
-2. Downstream performance uses a BP-calibrated effective task dimension.
-3. FA capacity loss is then tested with no extra FA-specific fitting parameter.
-
-Only add an effective FA burden coefficient
-
-\[
-K_{\mathrm{FA}}^{\mathrm{eff}}=\alpha K_{\mathrm{FA}}
-\]
-
-as an appendix ablation if needed. It improves fit but makes the theory look
-more ad hoc.