diff options
Diffstat (limited to 'notes/01_theory_notes.md')
| -rw-r--r-- | notes/01_theory_notes.md | 42 |
1 files changed, 42 insertions, 0 deletions
diff --git a/notes/01_theory_notes.md b/notes/01_theory_notes.md index 7a54374..b22e535 100644 --- a/notes/01_theory_notes.md +++ b/notes/01_theory_notes.md @@ -166,6 +166,48 @@ C_l\approx c/2, C_{\mathrm{all}}=\Theta(L). \] +## Candidate Corollary 2: Observed Capacity Distribution + +The threshold cost \(C_l(q)\) is deterministic once \(q\) is chosen. For empirical distribution matching, define the observed alignment surprisal: + +\[ +S_l += +-\log \Pr(Q_l'\ge Q_l), +\] + +where \(Q_l'\) is an independent draw from the same beta law as \(Q_l\). If: + +\[ +Q_l\sim \mathrm{Beta}\left(\frac12,\frac{D_l-1}{2}\right), +\] + +and the survival function is continuous, then by the probability integral transform: + +\[ +U_l=\Pr(Q_l'\ge Q_l)\sim \mathrm{Uniform}(0,1), +\] + +so: + +\[ +S_l=-\log U_l\sim \mathrm{Exp}(1). +\] + +For \(L\) independent feedback-aligned layers, the observed total capacity surprisal is: + +\[ +S_{1:L}=\sum_{l=1}^L S_l. +\] + +If the layerwise feedback matrices are independent, then: + +\[ +S_{1:L}\sim \mathrm{Gamma}(L,1). +\] + +This is useful because it predicts a full distribution rather than only a mean or bound. It also shows that after the beta-tail transform, the null distribution of observed capacity surprisal is dimension-free; dimension enters through the raw \(Q_l\) scale and the threshold cost \(C_l(q)\), not through the transformed surprisal law. + ## Candidate Theorem 2: Prior-Free Minimax Bound Let: |
