reports/distribution_matching_review.md


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149

# Distribution Matching Review

This report collects the current theory-vs-empirical plots.

## 1. Multilayer Capacity Distribution

Theory:

\[
S_l=-\log P(Q_l'\ge Q_l)\sim \mathrm{Exp}(1),
\qquad
\sum_{l=1}^L S_l\sim \mathrm{Gamma}(L,1).
\]

Large run:

- dimensions: `64, 256, 1024, 4096`
- layers: `1, 2, 4, 8, 16`
- samples per pair: `100000`
- max KS statistic: `0.0042004`
- mean absolute mean error: `0.0042410`

Key plots:

![capacity KS heatmap](../outputs/multilayer_capacity_distribution/ks_heatmap.png)

![D4096 L16 histogram](../outputs/multilayer_capacity_distribution/hist_D4096_L16.png)

![D4096 L16 QQ](../outputs/multilayer_capacity_distribution/qq_D4096_L16.png)

Judgment: this is strong distribution matching. The empirical histograms and QQ plots should be close enough for a paper figure.

## 2. Initialization Coverage Distribution

Theory:

\[
A(a)=a^\top M_\mu a
=
\frac{\sum_i \lambda_i G_i}{\sum_i G_i},
\qquad
G_i\sim\chi^2_1.
\]

Large run:

- dimension: `128`
- target samples: `100000`
- feedback samples: `100000`
- max nondegenerate KS: `0.00470`

Key plots:

![coverage histograms](../outputs/initialization_distribution_matching/coverage_histograms.png)

![spectra](../outputs/initialization_distribution_matching/population_vs_empirical_spectra.png)

![worst best coverage](../outputs/initialization_distribution_matching/worst_best_coverage.png)

![subspace QQ](../outputs/initialization_distribution_matching/qq_subspace.png)

![axis QQ](../outputs/initialization_distribution_matching/qq_axis.png)

![geometric axis QQ](../outputs/initialization_distribution_matching/qq_geometric_axis.png)

Judgment: this is strong distribution matching for non-isotropic schemes. Isotropic/rademacher are theoretically point masses at \(1/D\), so their QQ/KS is not the right diagnostic; mean and finite-sample spectrum spread are the right diagnostics there.

## 3. Trajectory Gap Distribution

Bridge predictor:

\[
\widehat{\delta\theta}_T(B)
=
-\eta
\sum_{t<T}
\left[
g_{\mathrm{FA}}(\theta_t^{\mathrm{BP}};B)
-
g_{\mathrm{BP}}(\theta_t^{\mathrm{BP}})
\right],
\]

\[
\widehat{\Delta L}_T(B)
=
L(\theta_T^{\mathrm{BP}}+\widehat{\delta\theta}_T(B))
-
L(\theta_T^{\mathrm{BP}}).
\]

### h16x16, 1000 runs

- empirical gap mean: `0.120109`
- bridge gap mean: `0.318266`
- raw KS: `0.917`
- standardized KS: `0.056`
- standardized p-value: `0.0869`
- moment-matched Wasserstein: `0.00362`
- paired Spearman: `0.49288`

Raw scale:

![h16 raw histogram](../outputs/trajectory_gap_distribution/h16x16_1000/gap_histogram_overlay.png)

![h16 raw QQ](../outputs/trajectory_gap_distribution/h16x16_1000/gap_qq_plot.png)

Moment-matched / standardized shape:

![h16 moment matched](../outputs/trajectory_gap_distribution/h16x16_1000/gap_histogram_moment_matched.png)

![h16 standardized QQ](../outputs/trajectory_gap_distribution/h16x16_1000/gap_standardized_qq_plot.png)

Paired seed ranking:

![h16 paired scatter](../outputs/trajectory_gap_distribution/h16x16_1000/paired_gap_scatter.png)

### h24x24, 500 runs

- empirical gap mean: `0.145215`
- bridge gap mean: `0.432834`
- raw KS: `0.944`
- standardized KS: `0.056`
- standardized p-value: `0.4135`
- moment-matched Wasserstein: `0.00356`
- paired Spearman: `0.39455`

Raw scale:

![h24 raw histogram](../outputs/trajectory_gap_distribution/h24x24_500/gap_histogram_overlay.png)

![h24 raw QQ](../outputs/trajectory_gap_distribution/h24x24_500/gap_qq_plot.png)

Moment-matched / standardized shape:

![h24 moment matched](../outputs/trajectory_gap_distribution/h24x24_500/gap_histogram_moment_matched.png)

![h24 standardized QQ](../outputs/trajectory_gap_distribution/h24x24_500/gap_standardized_qq_plot.png)

Paired seed ranking:

![h24 paired scatter](../outputs/trajectory_gap_distribution/h24x24_500/paired_gap_scatter.png)

Judgment:

- Raw bridge distribution is not sufficiently consistent: it overestimates the absolute gap scale by roughly `2-3x`.
- Standardized/moment-matched distribution shape is much more consistent.
- Current trajectory claim should be: the BP-path bridge captures distributional shape and seed ranking, but needs a response/curvature/contraction factor to predict absolute scale.