From 48f43ffa66c8af1b5aec123ebe9877bc3852ecd3 Mon Sep 17 00:00:00 2001 From: YurenHao0426 Date: Thu, 28 May 2026 22:54:53 -0500 Subject: Add prior-free minimax initialization simulation --- notes/02_experiment_notes.md | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) (limited to 'notes/02_experiment_notes.md') diff --git a/notes/02_experiment_notes.md b/notes/02_experiment_notes.md index ab69cd8..05b108b 100644 --- a/notes/02_experiment_notes.md +++ b/notes/02_experiment_notes.md @@ -259,3 +259,28 @@ Result: - chance-level max total cost: `18.3652` nats at width `128`, layers `16` This cleanly separates the fixed-threshold regime, where total cost scales like \(Ln^2\), from the chance-level regime, where per-layer cost is nearly width-independent. + +## Minimax Initialization Run Log + +Script: + +```bash +python scripts/minimax_initialization.py --dimension 32 --feedback-samples 20000 --target-samples 10000 --seed 11 --subspace-dim 4 --plot +``` + +Result: + +- minimax bound \(1/D\): `0.03125` +- isotropic \(\lambda_{\min}\): `0.029084138` +- rademacher \(\lambda_{\min}\): `0.028946927` +- anisotropic \(\lambda_{\min}\): `0.006149976` +- subspace \(\lambda_{\min}\): `0` +- axis \(\lambda_{\min}\): `0` + +Random-target means remain close to \(1/D\) for all distributions, but worst-case target coverage differs sharply: + +- isotropic and rademacher nearly equalize all target directions; +- anisotropic improves some directions while sacrificing others; +- subspace and axis initializations leave entire orthogonal directions uncovered. + +This empirically illustrates the prior-free minimax theorem: without target or weight prior information, anisotropic feedback cannot improve the worst-case angular bound. -- cgit v1.2.3