summaryrefslogtreecommitdiff
path: root/worldalign/field_anatomy.py
AgeCommit message (Collapse)Author
26 hoursTest the new instruments; fix an O(1/n) bias in the degree decompositionYurenHao0426
Four tests around today's additions. Two failed on first run and both were worth having. The degree decomposition left an O(1/n) residual on a field that is purely additive: excluding the diagonal makes the two-way design unbalanced, so one pass of row and column means does not remove a pure degree effect. Swept to convergence instead. At N=256 the correction moves the reported variance shares by under 0.001, so the refutation of the hubness hypothesis stands unchanged -- but the instrument that produced it now does what it claims. The other failure was the test's own scale: two random 16-dimensional subspaces of R^64 overlap above 0.7 by chance, which is why the real measurements are made at N=256 where the null sits at 1.0. Co-Authored-By: Claude <noreply@anthropic.com>
26 hoursRetire the correlation gate: the shared spectrum governs recoveryYurenHao0426
A controlled truncation refutes the project's central go/no-go rule. Projecting the recovering synthetic fields to rank r holds the field correlation at 0.902-0.929 while recovery moves 6.2% -> 12.9% -> 95.6% across ranks 4, 8, 16. A field past the supposed 0.9 threshold recovers 13%, so correlation neither predicts nor forbids recovery and the width of the shared spectrum is what moves it. The gate becomes a joint condition on correlation and shared width, measured by principal angles against a scene-shuffled null. Neither suffices alone: 18 shared directions at 0.508 fails, 11 at 0.902 fails. With the old gate retired, natural data was finally searched: 0.0000 against 0.0039 chance. The old verdict was right, its reasoning was not. Also closes route D by measurement. rho_IT ~ sqrt(4 log N / N) rises as N falls, and at N = 16 through 96 the deepest state a strong searcher reaches is deeper than the truth in 3/3 replicates at every size. Free gains: eigenvalue-weighted projection over a wide basis with 128-dim text vectors takes the correlation 0.656 -> 0.716 and shared width 10 -> 16. Hubness refuted as an inflation hypothesis. Moving the per-image segmentation eigendecomposition onto the GPU cut batch time from 130s to 1.9s. Co-Authored-By: Claude <noreply@anthropic.com>