diff options
Diffstat (limited to 'reports/distribution_matching_review.md')
| -rw-r--r-- | reports/distribution_matching_review.md | 149 |
1 files changed, 149 insertions, 0 deletions
diff --git a/reports/distribution_matching_review.md b/reports/distribution_matching_review.md new file mode 100644 index 0000000..833cf41 --- /dev/null +++ b/reports/distribution_matching_review.md @@ -0,0 +1,149 @@ +# Distribution Matching Review + +This report collects the current theory-vs-empirical plots. + +## 1. Multilayer Capacity Distribution + +Theory: + +\[ +S_l=-\log P(Q_l'\ge Q_l)\sim \mathrm{Exp}(1), +\qquad +\sum_{l=1}^L S_l\sim \mathrm{Gamma}(L,1). +\] + +Large run: + +- dimensions: `64, 256, 1024, 4096` +- layers: `1, 2, 4, 8, 16` +- samples per pair: `100000` +- max KS statistic: `0.0042004` +- mean absolute mean error: `0.0042410` + +Key plots: + + + + + + + +Judgment: this is strong distribution matching. The empirical histograms and QQ plots should be close enough for a paper figure. + +## 2. Initialization Coverage Distribution + +Theory: + +\[ +A(a)=a^\top M_\mu a += +\frac{\sum_i \lambda_i G_i}{\sum_i G_i}, +\qquad +G_i\sim\chi^2_1. +\] + +Large run: + +- dimension: `128` +- target samples: `100000` +- feedback samples: `100000` +- max nondegenerate KS: `0.00470` + +Key plots: + + + + + + + + + + + + + +Judgment: this is strong distribution matching for non-isotropic schemes. Isotropic/rademacher are theoretically point masses at \(1/D\), so their QQ/KS is not the right diagnostic; mean and finite-sample spectrum spread are the right diagnostics there. + +## 3. Trajectory Gap Distribution + +Bridge predictor: + +\[ +\widehat{\delta\theta}_T(B) += +-\eta +\sum_{t<T} +\left[ +g_{\mathrm{FA}}(\theta_t^{\mathrm{BP}};B) +- +g_{\mathrm{BP}}(\theta_t^{\mathrm{BP}}) +\right], +\] + +\[ +\widehat{\Delta L}_T(B) += +L(\theta_T^{\mathrm{BP}}+\widehat{\delta\theta}_T(B)) +- +L(\theta_T^{\mathrm{BP}}). +\] + +### h16x16, 1000 runs + +- empirical gap mean: `0.120109` +- bridge gap mean: `0.318266` +- raw KS: `0.917` +- standardized KS: `0.056` +- standardized p-value: `0.0869` +- moment-matched Wasserstein: `0.00362` +- paired Spearman: `0.49288` + +Raw scale: + + + + + +Moment-matched / standardized shape: + + + + + +Paired seed ranking: + + + +### h24x24, 500 runs + +- empirical gap mean: `0.145215` +- bridge gap mean: `0.432834` +- raw KS: `0.944` +- standardized KS: `0.056` +- standardized p-value: `0.4135` +- moment-matched Wasserstein: `0.00356` +- paired Spearman: `0.39455` + +Raw scale: + + + + + +Moment-matched / standardized shape: + + + + + +Paired seed ranking: + + + +Judgment: + +- Raw bridge distribution is not sufficiently consistent: it overestimates the absolute gap scale by roughly `2-3x`. +- Standardized/moment-matched distribution shape is much more consistent. +- Current trajectory claim should be: the BP-path bridge captures distributional shape and seed ranking, but needs a response/curvature/contraction factor to predict absolute scale. + |