Retire the correlation gate: the shared spectrum governs recovery

A controlled truncation refutes the project's central go/no-go rule.
Projecting the recovering synthetic fields to rank r holds the field
correlation at 0.902-0.929 while recovery moves 6.2% -> 12.9% -> 95.6%
across ranks 4, 8, 16. A field past the supposed 0.9 threshold recovers
13%, so correlation neither predicts nor forbids recovery and the width
of the shared spectrum is what moves it.

The gate becomes a joint condition on correlation and shared width,
measured by principal angles against a scene-shuffled null. Neither
suffices alone: 18 shared directions at 0.508 fails, 11 at 0.902 fails.

With the old gate retired, natural data was finally searched: 0.0000
against 0.0039 chance. The old verdict was right, its reasoning was not.

Also closes route D by measurement. rho_IT ~ sqrt(4 log N / N) rises as N
falls, and at N = 16 through 96 the deepest state a strong searcher
reaches is deeper than the truth in 3/3 replicates at every size.

Free gains: eigenvalue-weighted projection over a wide basis with
128-dim text vectors takes the correlation 0.656 -> 0.716 and shared
width 10 -> 16. Hubness refuted as an inflation hypothesis.

Moving the per-image segmentation eigendecomposition onto the GPU cut
batch time from 130s to 1.9s.

Co-Authored-By: Claude <noreply@anthropic.com>

1 files changed, 351 insertions, 0 deletions

diff --git a/artifacts/vg_5k/shared_rank_degree.json b/artifacts/vg_5k/shared_rank_degree.json
new file mode 100644
index 0000000..7940d64
--- /dev/null
+++ b/artifacts/vg_5k/shared_rank_degree.json
@@ -0,0 +1,351 @@
+{
+  "protocol": "Effective rank from the participation ratio of each field's spectrum; shared directions from principal angles between the two leading eigenspaces. No pairing enters beyond the alignment the fields already carry.",
+  "width": 32,
+  "rows": [
+    {
+      "label": "degree2-best",
+      "size": 256,
+      "correlation": 0.7157944505313559,
+      "visual_effective_rank": 39.69132496762981,
+      "text_effective_rank": 47.98422953390366,
+      "shared_directions_cos_gt_0.7": 16,
+      "shuffled_null": 1.0,
+      "shared_above_null": 15.0,
+      "principal_cosines_top8": [
+        0.987,
+        0.978,
+        0.96,
+        0.941,
+        0.934,
+        0.922,
+        0.899,
+        0.89
+      ],
+      "spectrum_profile": [
+        {
+          "index": 0,
+          "best_text_overlap": 0.8406807387214822
+        },
+        {
+          "index": 1,
+          "best_text_overlap": 0.6748753041856854
+        },
+        {
+          "index": 2,
+          "best_text_overlap": 0.7174150319587717
+        },
+        {
+          "index": 3,
+          "best_text_overlap": 0.4742164704027202
+        },
+        {
+          "index": 4,
+          "best_text_overlap": 0.7105187496542062
+        },
+        {
+          "index": 5,
+          "best_text_overlap": 0.5435228035336661
+        },
+        {
+          "index": 6,
+          "best_text_overlap": 0.5173316630294161
+        },
+        {
+          "index": 7,
+          "best_text_overlap": 0.3720956303455928
+        },
+        {
+          "index": 8,
+          "best_text_overlap": 0.4031225977235423
+        },
+        {
+          "index": 9,
+          "best_text_overlap": 0.5909531802492844
+        },
+        {
+          "index": 10,
+          "best_text_overlap": 0.373881786434362
+        },
+        {
+          "index": 11,
+          "best_text_overlap": 0.2816693575587397
+        },
+        {
+          "index": 12,
+          "best_text_overlap": 0.21822240451244762
+        },
+        {
+          "index": 13,
+          "best_text_overlap": 0.3929212823566225
+        },
+        {
+          "index": 14,
+          "best_text_overlap": 0.4024276537497661
+        },
+        {
+          "index": 15,
+          "best_text_overlap": 0.2659834432053085
+        }
+      ]
+    },
+    {
+      "label": "degree3-w8",
+      "size": 256,
+      "correlation": 0.717450184813228,
+      "visual_effective_rank": 40.67303880139185,
+      "text_effective_rank": 47.11594894154097,
+      "shared_directions_cos_gt_0.7": 16,
+      "shuffled_null": 1.0,
+      "shared_above_null": 15.0,
+      "principal_cosines_top8": [
+        0.988,
+        0.978,
+        0.961,
+        0.944,
+        0.936,
+        0.924,
+        0.901,
+        0.886
+      ],
+      "spectrum_profile": [
+        {
+          "index": 0,
+          "best_text_overlap": 0.8348375430905127
+        },
+        {
+          "index": 1,
+          "best_text_overlap": 0.6837720540848308
+        },
+        {
+          "index": 2,
+          "best_text_overlap": 0.7155802705177406
+        },
+        {
+          "index": 3,
+          "best_text_overlap": 0.5058562446867387
+        },
+        {
+          "index": 4,
+          "best_text_overlap": 0.7663355948751955
+        },
+        {
+          "index": 5,
+          "best_text_overlap": 0.516380442780415
+        },
+        {
+          "index": 6,
+          "best_text_overlap": 0.554955451295094
+        },
+        {
+          "index": 7,
+          "best_text_overlap": 0.3516466638329569
+        },
+        {
+          "index": 8,
+          "best_text_overlap": 0.5186308948947995
+        },
+        {
+          "index": 9,
+          "best_text_overlap": 0.5015555070809197
+        },
+        {
+          "index": 10,
+          "best_text_overlap": 0.3700340337533245
+        },
+        {
+          "index": 11,
+          "best_text_overlap": 0.34258025998594904
+        },
+        {
+          "index": 12,
+          "best_text_overlap": 0.256732226634933
+        },
+        {
+          "index": 13,
+          "best_text_overlap": 0.4359760369195398
+        },
+        {
+          "index": 14,
+          "best_text_overlap": 0.32969959689961453
+        },
+        {
+          "index": 15,
+          "best_text_overlap": 0.2530743072716043
+        }
+      ]
+    },
+    {
+      "label": "degree3-w12",
+      "size": 256,
+      "correlation": 0.7164530320598383,
+      "visual_effective_rank": 42.77448656306514,
+      "text_effective_rank": 47.90052292048341,
+      "shared_directions_cos_gt_0.7": 16,
+      "shuffled_null": 1.0,
+      "shared_above_null": 15.0,
+      "principal_cosines_top8": [
+        0.988,
+        0.978,
+        0.961,
+        0.946,
+        0.937,
+        0.929,
+        0.91,
+        0.889
+      ],
+      "spectrum_profile": [
+        {
+          "index": 0,
+          "best_text_overlap": 0.8277435616658919
+        },
+        {
+          "index": 1,
+          "best_text_overlap": 0.6779609268778297
+        },
+        {
+          "index": 2,
+          "best_text_overlap": 0.7219010238018317
+        },
+        {
+          "index": 3,
+          "best_text_overlap": 0.5055290038432999
+        },
+        {
+          "index": 4,
+          "best_text_overlap": 0.7718721615385675
+        },
+        {
+          "index": 5,
+          "best_text_overlap": 0.5256873621550153
+        },
+        {
+          "index": 6,
+          "best_text_overlap": 0.5367136087431718
+        },
+        {
+          "index": 7,
+          "best_text_overlap": 0.36680812886106473
+        },
+        {
+          "index": 8,
+          "best_text_overlap": 0.5387792531732701
+        },
+        {
+          "index": 9,
+          "best_text_overlap": 0.5550666365558536
+        },
+        {
+          "index": 10,
+          "best_text_overlap": 0.2891791715134958
+        },
+        {
+          "index": 11,
+          "best_text_overlap": 0.2984192540709536
+        },
+        {
+          "index": 12,
+          "best_text_overlap": 0.28170926274492825
+        },
+        {
+          "index": 13,
+          "best_text_overlap": 0.4002691992367436
+        },
+        {
+          "index": 14,
+          "best_text_overlap": 0.31596068490447127
+        },
+        {
+          "index": 15,
+          "best_text_overlap": 0.26924174791563826
+        }
+      ]
+    },
+    {
+      "label": "degree3-w20",
+      "size": 256,
+      "correlation": 0.7171326909156863,
+      "visual_effective_rank": 46.709791939402166,
+      "text_effective_rank": 50.02596883001826,
+      "shared_directions_cos_gt_0.7": 17,
+      "shuffled_null": 1.0,
+      "shared_above_null": 16.0,
+      "principal_cosines_top8": [
+        0.988,
+        0.978,
+        0.962,
+        0.945,
+        0.939,
+        0.931,
+        0.911,
+        0.894
+      ],
+      "spectrum_profile": [
+        {
+          "index": 0,
+          "best_text_overlap": 0.8231142857010968
+        },
+        {
+          "index": 1,
+          "best_text_overlap": 0.6748733981742054
+        },
+        {
+          "index": 2,
+          "best_text_overlap": 0.7180139533448364
+        },
+        {
+          "index": 3,
+          "best_text_overlap": 0.5179878165995193
+        },
+        {
+          "index": 4,
+          "best_text_overlap": 0.7736927465190133
+        },
+        {
+          "index": 5,
+          "best_text_overlap": 0.5414967531898147
+        },
+        {
+          "index": 6,
+          "best_text_overlap": 0.5416571648246705
+        },
+        {
+          "index": 7,
+          "best_text_overlap": 0.3600170379372908
+        },
+        {
+          "index": 8,
+          "best_text_overlap": 0.5145850503977925
+        },
+        {
+          "index": 9,
+          "best_text_overlap": 0.5683887253705958
+        },
+        {
+          "index": 10,
+          "best_text_overlap": 0.30798470685067697
+        },
+        {
+          "index": 11,
+          "best_text_overlap": 0.31660436348552895
+        },
+        {
+          "index": 12,
+          "best_text_overlap": 0.2381351557071557
+        },
+        {
+          "index": 13,
+          "best_text_overlap": 0.44012504821073994
+        },
+        {
+          "index": 14,
+          "best_text_overlap": 0.35872551158185606
+        },
+        {
+          "index": 15,
+          "best_text_overlap": 0.2717148785929642
+        }
+      ]
+    }
+  ],
+  "reading": "The narrow modality is the one to upgrade. A shared count far below both effective ranks means each side is rich but they are rich about different things."
+}
\ No newline at end of file