From 48f43ffa66c8af1b5aec123ebe9877bc3852ecd3 Mon Sep 17 00:00:00 2001
From: YurenHao0426 <Blackhao0426@gmail.com>
Date: Thu, 28 May 2026 22:54:53 -0500
Subject: Add prior-free minimax initialization simulation

---
 notes/01_theory_notes.md | 88 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 88 insertions(+)

(limited to 'notes/01_theory_notes.md')

diff --git a/notes/01_theory_notes.md b/notes/01_theory_notes.md
index c0ac8fe..d524000 100644
--- a/notes/01_theory_notes.md
+++ b/notes/01_theory_notes.md
@@ -215,6 +215,94 @@ Thus:
 
 Isotropic initialization attains this bound.
 
+### Proof Sketch
+
+The proof only needs second moments. For any feedback initialization \(\mu\), define:
+
+\[
+M_\mu=\mathbb E_\mu[\hat b\hat b^\top].
+\]
+
+Since \(\|\hat b\|=1\):
+
+\[
+\operatorname{tr}M_\mu
+=
+\mathbb E_\mu[\operatorname{tr}(\hat b\hat b^\top)]
+=
+\mathbb E_\mu[\|\hat b\|^2]
+=1.
+\]
+
+For any fixed target direction \(a\):
+
+\[
+\mathbb E_\mu[(a^\top \hat b)^2]
+=
+\mathbb E_\mu[a^\top \hat b\hat b^\top a]
+=
+a^\top M_\mu a.
+\]
+
+The worst-case target direction is the minimum-eigenvalue direction of \(M_\mu\):
+
+\[
+\inf_{\|a\|=1} a^\top M_\mu a
+=
+\lambda_{\min}(M_\mu).
+\]
+
+Because the average eigenvalue is \(1/D\):
+
+\[
+\lambda_{\min}(M_\mu)
+\le
+\frac1D.
+\]
+
+Hence:
+
+\[
+\inf_{\|a\|=1}
+\mathbb E_\mu[(a^\top \hat b)^2]
+\le
+\frac1D
+\]
+
+for every initialization \(\mu\). If \(\mu\) is isotropic, then:
+
+\[
+M_\mu=\frac1D I,
+\]
+
+so every direction has:
+
+\[
+a^\top M_\mu a=\frac1D.
+\]
+
+Thus:
+
+\[
+\sup_\mu
+\inf_{\|a\|=1}
+\mathbb E_\mu[(a^\top \hat b)^2]
+=
+\frac1D.
+\]
+
+This is a worst-case theorem. If the target direction is itself uniformly random, then:
+
+\[
+\mathbb E_a[a^\top M_\mu a]
+=
+\frac{\operatorname{tr}M_\mu}{D}
+=
+\frac1D
+\]
+
+for every \(\mu\). Therefore prior-free average-case alignment over uniform targets cannot distinguish initializations by their mean; the distinction is worst-case coverage, tail behavior, and conditioning.
+
 ## Prior-Aware Corollary
 
 If target directions have prior covariance:
-- 
cgit v1.2.3