diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-06-02 15:13:04 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-06-02 15:13:04 -0500 |
| commit | daf0a6dc795235864d8279908320801f3c7b66b5 (patch) | |
| tree | ebe59a8768889cf5430c6a4ec142a2e3a7693a1b /notes/07_minimal_transition_theory.md | |
| parent | b436e6c033c2c3b37d2ee26efe0dbe28381117dc (diff) | |
Replace transition theory with tangent capacity
Diffstat (limited to 'notes/07_minimal_transition_theory.md')
| -rw-r--r-- | notes/07_minimal_transition_theory.md | 104 |
1 files changed, 0 insertions, 104 deletions
diff --git a/notes/07_minimal_transition_theory.md b/notes/07_minimal_transition_theory.md deleted file mode 100644 index ced7e61..0000000 --- a/notes/07_minimal_transition_theory.md +++ /dev/null @@ -1,104 +0,0 @@ -# Minimal Transition Theory - -The cleanest correction is not to add a separate FA burden coefficient first. -Start by calibrating the task dimension from BP alone. - -## Problem - -The hard theory used: - -\[ -d_{\mathrm{task}}=Nn_L=512. -\] - -With this value, the smooth theory peak is near: - -\[ -M_{\mathrm{FA}}\approx -273, -\] - -while 256 empirical FA trajectories peak at: - -\[ -M_{\mathrm{FA}}=-318. -\] - -## BP-Only Calibration - -Fit the BP train curve with: - -\[ -L_{\mathrm{BP}}(P) -\approx -s\cdot \operatorname{softplus}_{\tau}(d_{\mathrm{task}}^{\mathrm{eff}}-P). -\] - -The BP-only fit gives: - -\[ -d_{\mathrm{task}}^{\mathrm{eff}}\approx 437, -\qquad -\tau\approx 50. -\] - -So BP is stronger than the naive parameter count suggests: - -\[ -d_{\mathrm{task}}^{\mathrm{eff}} -\approx -0.85\,Nn_L. -\] - -## Minimal FA Prediction - -Use the same hard FA burden: - -\[ -K_{\mathrm{FA}}=\sum_l(D_l-1), -\] - -but replace the task dimension: - -\[ -M_{\mathrm{FA}}^{\mathrm{min}} -= -P-K_{\mathrm{FA}}-d_{\mathrm{task}}^{\mathrm{eff}}. -\] - -Equivalently, when plotted on the original hard-margin axis - -\[ -P-K_{\mathrm{FA}}-Nn_L, -\] - -the theory shifts left by: - -\[ -Nn_L-d_{\mathrm{task}}^{\mathrm{eff}} -\approx -75. -\] - -This explains the observed left shift without introducing a fitted FA burden -coefficient. - -Generated figure: - -`outputs/downstream_capacity_random_main_fast/capacity_transition_bp_calibrated_p2_256.png` - -## Recommendation - -Use this as the main downstream theory: - -1. Matrix/capacity theory remains exact and uses the hard/log-volume definitions. -2. Downstream performance uses a BP-calibrated effective task dimension. -3. FA capacity loss is then tested with no extra FA-specific fitting parameter. - -Only add an effective FA burden coefficient - -\[ -K_{\mathrm{FA}}^{\mathrm{eff}}=\alpha K_{\mathrm{FA}} -\] - -as an appendix ablation if needed. It improves fit but makes the theory look -more ad hoc. |
