Retire the correlation gate: the shared spectrum governs recovery

A controlled truncation refutes the project's central go/no-go rule.
Projecting the recovering synthetic fields to rank r holds the field
correlation at 0.902-0.929 while recovery moves 6.2% -> 12.9% -> 95.6%
across ranks 4, 8, 16. A field past the supposed 0.9 threshold recovers
13%, so correlation neither predicts nor forbids recovery and the width
of the shared spectrum is what moves it.

The gate becomes a joint condition on correlation and shared width,
measured by principal angles against a scene-shuffled null. Neither
suffices alone: 18 shared directions at 0.508 fails, 11 at 0.902 fails.

With the old gate retired, natural data was finally searched: 0.0000
against 0.0039 chance. The old verdict was right, its reasoning was not.

Also closes route D by measurement. rho_IT ~ sqrt(4 log N / N) rises as N
falls, and at N = 16 through 96 the deepest state a strong searcher
reaches is deeper than the truth in 3/3 replicates at every size.

Free gains: eigenvalue-weighted projection over a wide basis with
128-dim text vectors takes the correlation 0.656 -> 0.716 and shared
width 10 -> 16. Hubness refuted as an inflation hypothesis.

Moving the per-image segmentation eigendecomposition onto the GPU cut
batch time from 130s to 1.9s.

Co-Authored-By: Claude <noreply@anthropic.com>

1 files changed, 343 insertions, 0 deletions

diff --git a/artifacts/vg_5k/shared_rank_segments.json b/artifacts/vg_5k/shared_rank_segments.json
new file mode 100644
index 0000000..7408bd4
--- /dev/null
+++ b/artifacts/vg_5k/shared_rank_segments.json
@@ -0,0 +1,343 @@
+{
+  "protocol": "Effective rank from the participation ratio of each field's spectrum; shared directions from principal angles between the two leading eigenspaces. No pairing enters beyond the alignment the fields already carry.",
+  "width": 32,
+  "rows": [
+    {
+      "label": "synth-RECOVERS",
+      "size": 256,
+      "correlation": 0.9284809883663888,
+      "visual_effective_rank": 14.50854049599196,
+      "text_effective_rank": 11.832081486516831,
+      "shared_directions_cos_gt_0.7": 27,
+      "principal_cosines_top8": [
+        1.0,
+        1.0,
+        1.0,
+        1.0,
+        1.0,
+        0.999,
+        0.999,
+        0.999
+      ],
+      "spectrum_profile": [
+        {
+          "index": 0,
+          "best_text_overlap": 0.9988184979219944
+        },
+        {
+          "index": 1,
+          "best_text_overlap": 0.9038254548424662
+        },
+        {
+          "index": 2,
+          "best_text_overlap": 0.7738637793117231
+        },
+        {
+          "index": 3,
+          "best_text_overlap": 0.7898924505967239
+        },
+        {
+          "index": 4,
+          "best_text_overlap": 0.9633572761985347
+        },
+        {
+          "index": 5,
+          "best_text_overlap": 0.9667594083538564
+        },
+        {
+          "index": 6,
+          "best_text_overlap": 0.8581068112833727
+        },
+        {
+          "index": 7,
+          "best_text_overlap": 0.7720408457258224
+        },
+        {
+          "index": 8,
+          "best_text_overlap": 0.9451869286586472
+        },
+        {
+          "index": 9,
+          "best_text_overlap": 0.9346387523699597
+        },
+        {
+          "index": 10,
+          "best_text_overlap": 0.9744620235608439
+        },
+        {
+          "index": 11,
+          "best_text_overlap": 0.8287855141725994
+        },
+        {
+          "index": 12,
+          "best_text_overlap": 0.8450929971524885
+        },
+        {
+          "index": 13,
+          "best_text_overlap": 0.7475087759782241
+        },
+        {
+          "index": 14,
+          "best_text_overlap": 0.5363062788775304
+        },
+        {
+          "index": 15,
+          "best_text_overlap": 0.5851587348701632
+        }
+      ]
+    },
+    {
+      "label": "nat-6seg",
+      "size": 256,
+      "correlation": 0.6922159706308013,
+      "visual_effective_rank": 39.69132496762981,
+      "text_effective_rank": 31.7662539087451,
+      "shared_directions_cos_gt_0.7": 15,
+      "principal_cosines_top8": [
+        0.989,
+        0.978,
+        0.956,
+        0.938,
+        0.916,
+        0.892,
+        0.881,
+        0.87
+      ],
+      "spectrum_profile": [
+        {
+          "index": 0,
+          "best_text_overlap": 0.7532219338133863
+        },
+        {
+          "index": 1,
+          "best_text_overlap": 0.618182152081914
+        },
+        {
+          "index": 2,
+          "best_text_overlap": 0.6411572047678387
+        },
+        {
+          "index": 3,
+          "best_text_overlap": 0.5609614131227192
+        },
+        {
+          "index": 4,
+          "best_text_overlap": 0.5786521313263311
+        },
+        {
+          "index": 5,
+          "best_text_overlap": 0.6212040366073345
+        },
+        {
+          "index": 6,
+          "best_text_overlap": 0.6014409101136969
+        },
+        {
+          "index": 7,
+          "best_text_overlap": 0.3241154791286174
+        },
+        {
+          "index": 8,
+          "best_text_overlap": 0.33772536144199544
+        },
+        {
+          "index": 9,
+          "best_text_overlap": 0.46886651280081304
+        },
+        {
+          "index": 10,
+          "best_text_overlap": 0.339879585332424
+        },
+        {
+          "index": 11,
+          "best_text_overlap": 0.2697704959742108
+        },
+        {
+          "index": 12,
+          "best_text_overlap": 0.3089464131279491
+        },
+        {
+          "index": 13,
+          "best_text_overlap": 0.34824951852897246
+        },
+        {
+          "index": 14,
+          "best_text_overlap": 0.26546032832701905
+        },
+        {
+          "index": 15,
+          "best_text_overlap": 0.17108445671033506
+        }
+      ]
+    },
+    {
+      "label": "nat-16seg",
+      "size": 256,
+      "correlation": 0.6794444345566248,
+      "visual_effective_rank": 40.32463000927306,
+      "text_effective_rank": 31.7662539087451,
+      "shared_directions_cos_gt_0.7": 14,
+      "principal_cosines_top8": [
+        0.991,
+        0.979,
+        0.955,
+        0.937,
+        0.917,
+        0.897,
+        0.882,
+        0.87
+      ],
+      "spectrum_profile": [
+        {
+          "index": 0,
+          "best_text_overlap": 0.7366894502381554
+        },
+        {
+          "index": 1,
+          "best_text_overlap": 0.6371340906520718
+        },
+        {
+          "index": 2,
+          "best_text_overlap": 0.6670233369588643
+        },
+        {
+          "index": 3,
+          "best_text_overlap": 0.7305683273676973
+        },
+        {
+          "index": 4,
+          "best_text_overlap": 0.5178189484095195
+        },
+        {
+          "index": 5,
+          "best_text_overlap": 0.6108081877127542
+        },
+        {
+          "index": 6,
+          "best_text_overlap": 0.5743712616080214
+        },
+        {
+          "index": 7,
+          "best_text_overlap": 0.30576250785088804
+        },
+        {
+          "index": 8,
+          "best_text_overlap": 0.31705130049377683
+        },
+        {
+          "index": 9,
+          "best_text_overlap": 0.434975496479508
+        },
+        {
+          "index": 10,
+          "best_text_overlap": 0.2627438939568967
+        },
+        {
+          "index": 11,
+          "best_text_overlap": 0.22483152594639236
+        },
+        {
+          "index": 12,
+          "best_text_overlap": 0.24208088194854407
+        },
+        {
+          "index": 13,
+          "best_text_overlap": 0.2850328753356361
+        },
+        {
+          "index": 14,
+          "best_text_overlap": 0.24894409456686264
+        },
+        {
+          "index": 15,
+          "best_text_overlap": 0.3923832160325988
+        }
+      ]
+    },
+    {
+      "label": "nat-oracle-regions",
+      "size": 256,
+      "correlation": 0.702748890720233,
+      "visual_effective_rank": 39.32779841860138,
+      "text_effective_rank": 31.7662539087451,
+      "shared_directions_cos_gt_0.7": 15,
+      "principal_cosines_top8": [
+        0.99,
+        0.978,
+        0.959,
+        0.939,
+        0.921,
+        0.899,
+        0.89,
+        0.876
+      ],
+      "spectrum_profile": [
+        {
+          "index": 0,
+          "best_text_overlap": 0.7416685704809811
+        },
+        {
+          "index": 1,
+          "best_text_overlap": 0.6591035296451755
+        },
+        {
+          "index": 2,
+          "best_text_overlap": 0.6730514886891261
+        },
+        {
+          "index": 3,
+          "best_text_overlap": 0.5813764120097434
+        },
+        {
+          "index": 4,
+          "best_text_overlap": 0.5492379792963318
+        },
+        {
+          "index": 5,
+          "best_text_overlap": 0.6616657410048805
+        },
+        {
+          "index": 6,
+          "best_text_overlap": 0.5607722465245946
+        },
+        {
+          "index": 7,
+          "best_text_overlap": 0.3913414443532297
+        },
+        {
+          "index": 8,
+          "best_text_overlap": 0.3431217610653001
+        },
+        {
+          "index": 9,
+          "best_text_overlap": 0.39446274885223964
+        },
+        {
+          "index": 10,
+          "best_text_overlap": 0.2624025040480312
+        },
+        {
+          "index": 11,
+          "best_text_overlap": 0.31050951747572164
+        },
+        {
+          "index": 12,
+          "best_text_overlap": 0.37391245736141376
+        },
+        {
+          "index": 13,
+          "best_text_overlap": 0.2550412331810302
+        },
+        {
+          "index": 14,
+          "best_text_overlap": 0.29254528024572746
+        },
+        {
+          "index": 15,
+          "best_text_overlap": 0.3608253612065695
+        }
+      ]
+    }
+  ],
+  "reading": "The narrow modality is the one to upgrade. A shared count far below both effective ranks means each side is rich but they are rich about different things."
+}
\ No newline at end of file