diff options
Diffstat (limited to 'notes/02_experiment_notes.md')
| -rw-r--r-- | notes/02_experiment_notes.md | 25 |
1 files changed, 25 insertions, 0 deletions
diff --git a/notes/02_experiment_notes.md b/notes/02_experiment_notes.md index ab69cd8..05b108b 100644 --- a/notes/02_experiment_notes.md +++ b/notes/02_experiment_notes.md @@ -259,3 +259,28 @@ Result: - chance-level max total cost: `18.3652` nats at width `128`, layers `16` This cleanly separates the fixed-threshold regime, where total cost scales like \(Ln^2\), from the chance-level regime, where per-layer cost is nearly width-independent. + +## Minimax Initialization Run Log + +Script: + +```bash +python scripts/minimax_initialization.py --dimension 32 --feedback-samples 20000 --target-samples 10000 --seed 11 --subspace-dim 4 --plot +``` + +Result: + +- minimax bound \(1/D\): `0.03125` +- isotropic \(\lambda_{\min}\): `0.029084138` +- rademacher \(\lambda_{\min}\): `0.028946927` +- anisotropic \(\lambda_{\min}\): `0.006149976` +- subspace \(\lambda_{\min}\): `0` +- axis \(\lambda_{\min}\): `0` + +Random-target means remain close to \(1/D\) for all distributions, but worst-case target coverage differs sharply: + +- isotropic and rademacher nearly equalize all target directions; +- anisotropic improves some directions while sacrificing others; +- subspace and axis initializations leave entire orthogonal directions uncovered. + +This empirically illustrates the prior-free minimax theorem: without target or weight prior information, anisotropic feedback cannot improve the worst-case angular bound. |