diff options
Diffstat (limited to 'notes/05_downstream_random_capacity_results.md')
| -rw-r--r-- | notes/05_downstream_random_capacity_results.md | 78 |
1 files changed, 78 insertions, 0 deletions
diff --git a/notes/05_downstream_random_capacity_results.md b/notes/05_downstream_random_capacity_results.md new file mode 100644 index 0000000..b6b0e92 --- /dev/null +++ b/notes/05_downstream_random_capacity_results.md @@ -0,0 +1,78 @@ +# Downstream Random-Capacity Results + +## Main Run + +Output directory: + +`outputs/downstream_capacity_random_main_fast` + +Task: + +\[ +x_i\sim \mathcal N(0,I_{16}), +\qquad +y_i\sim \mathcal N(0,I_4), +\qquad +N=128. +\] + +Task dimension: + +\[ +N n_L=512. +\] + +Model: + +\[ +16\to n\to n\to 4. +\] + +Theory: + +\[ +P(n)=16n+n^2+4n, +\] + +\[ +K_{\mathrm{FA}}(n)=(n^2-1)+(4n-1), +\] + +\[ +M_{\mathrm{BP}}=P-Nn_L, +\qquad +M_{\mathrm{FA}}=P-K_{\mathrm{FA}}-Nn_L. +\] + +Prediction: + +- BP threshold at width \(16\). +- FA threshold at width \(32\). + +Trajectory count: + +- BP: \(4\) seeds per width. +- FA: \(4\times 12=48\) trajectories per width. +- Widths: \(8,12,16,24,32,48,64,96\). + +## Summary + +| width | BP train | FA train | gap | BP margin | FA margin | +|---:|---:|---:|---:|---:|---:| +| 8 | 0.7088 | 1.064 | 0.3555 | -288 | -382 | +| 12 | 0.2252 | 0.6141 | 0.3889 | -128 | -318 | +| 16 | 0.00743 | 0.2948 | 0.2873 | 64 | -254 | +| 24 | 0.000129 | 0.0346 | 0.0345 | 544 | -126 | +| 32 | 0.000007 | 0.0108 | 0.0108 | 1152 | 2 | +| 48 | 0.000007 | 0.00392 | 0.00391 | 2752 | 258 | +| 64 | 0.000046 | 0.00189 | 0.00185 | 4864 | 514 | +| 96 | 0.000250 | 0.00231 | 0.00206 | 10624 | 1026 | + +## Main Figures + +- `outputs/downstream_capacity_random_main_fast/train_mse_trajectory_distribution.png` +- `outputs/downstream_capacity_random_main_fast/train_gap_trajectory_distribution_vs_fa_margin.png` +- `outputs/downstream_capacity_random_main_fast/bp_fa_train_mse_vs_width.png` +- `outputs/downstream_capacity_random_main_fast/train_gap_vs_fa_capacity_margin.png` + +The cleanest figure is `train_gap_trajectory_distribution_vs_fa_margin.png`: the FA/BP train gap is large when \(M_{\mathrm{FA}}<0\), drops sharply near \(M_{\mathrm{FA}}=0\), and remains small for positive FA margin. |