diff options
Diffstat (limited to 'notes')
| -rw-r--r-- | notes/02_experiment_notes.md | 58 | ||||
| -rw-r--r-- | notes/03_paper_outline.md | 13 |
2 files changed, 71 insertions, 0 deletions
diff --git a/notes/02_experiment_notes.md b/notes/02_experiment_notes.md index 2fad18f..c7e7168 100644 --- a/notes/02_experiment_notes.md +++ b/notes/02_experiment_notes.md @@ -385,6 +385,64 @@ Random-target means remain close to \(1/D\) for all distributions, but worst-cas This empirically illustrates the prior-free minimax theorem: without target or weight prior information, anisotropic feedback cannot improve the worst-case angular bound. +## Initialization Coverage Distribution Matching Run Log + +Script: + +```bash +python scripts/initialization_distribution_matching.py --dimension 128 --target-samples 100000 --feedback-samples 100000 --batch-size 8192 --seed 2026 --subspace-dim 8 --anisotropy 64 --plot +``` + +Theory: + +For a feedback initialization distribution \(\mu\): + +\[ +M_\mu=\mathbb E[\hat b\hat b^\top]. +\] + +For a random target direction \(a\): + +\[ +A(a)=a^\top M_\mu a. +\] + +If \(\lambda_i\) are eigenvalues of \(M_\mu\), then: + +\[ +A(a) += +\frac{\sum_i \lambda_i G_i}{\sum_i G_i}, +\qquad +G_i\sim\chi^2_1. +\] + +Setup: + +- dimension \(D=128\) +- target samples: `100000` +- feedback samples for empirical \(M_\mu\): `100000` +- schemes: `isotropic`, `rademacher`, `subspace`, `axis`, `geometric_axis` +- subspace dimension: `8` +- geometric axis anisotropy: `64` + +Results: + +| scheme | population \(\lambda_{\min}\) | empirical \(\lambda_{\min}\) | predicted mean | empirical mean | predicted std | empirical std | KS | +|---|---:|---:|---:|---:|---:|---:|---:| +| `isotropic` | `0.0078125` | `0.0072928` | `0.0078125` | `0.0078124` | `~0` | `0.0000346` | `nan` | +| `rademacher` | `0.0078125` | `0.0072982` | `0.0078125` | `0.0078125` | `~0` | `0.0000342` | `nan` | +| `subspace` | `0` | `0` | `0.0077967` | `0.0078031` | `0.0037405` | `0.0037335` | `0.00299` | +| `axis` | `0` | `0` | `0.0078396` | `0.0078514` | `0.0109834` | `0.0109230` | `0.00351` | +| `geometric_axis` | `0.0005111` | `0.0004100` | `0.0078116` | `0.0078155` | `0.0010441` | `0.0010425` | `0.00470` | + +Interpretation: + +- All schemes have mean coverage near \(1/D\), as expected from averaging over uniformly random targets. +- Isotropic and rademacher have population point-mass coverage at \(1/D\); empirical spread comes from finite-sample \(M_\mu\) estimation noise, so KS against a point mass is not meaningful. +- Non-isotropic schemes match the predicted target-coverage distribution tightly, with max nondegenerate KS `0.00470`. +- The distribution shape, not just the mean, exposes the tradeoff: subspace and axis schemes keep the same average but create zero-coverage worst-case directions. + ## Functional Capacity Overlap Run Log Script: diff --git a/notes/03_paper_outline.md b/notes/03_paper_outline.md index 328871f..6055722 100644 --- a/notes/03_paper_outline.md +++ b/notes/03_paper_outline.md @@ -128,6 +128,19 @@ Prior-aware corollary: \operatorname{tr}(\Sigma_A M_\mu). \] +Distributional validation target: + +\[ +A(a)=a^\top M_\mu a += +\frac{\sum_i \lambda_i G_i}{\sum_i G_i}, +\qquad +G_i\sim\chi^2_1. +\] + +Use this to compare predicted target-coverage distributions for isotropic, +subspace, axis, and anisotropic feedback initializations. + ## 7. Trajectory Bridge Introduce local approximation: |