Retire the correlation gate: the shared spectrum governs recovery

A controlled truncation refutes the project's central go/no-go rule.
Projecting the recovering synthetic fields to rank r holds the field
correlation at 0.902-0.929 while recovery moves 6.2% -> 12.9% -> 95.6%
across ranks 4, 8, 16. A field past the supposed 0.9 threshold recovers
13%, so correlation neither predicts nor forbids recovery and the width
of the shared spectrum is what moves it.

The gate becomes a joint condition on correlation and shared width,
measured by principal angles against a scene-shuffled null. Neither
suffices alone: 18 shared directions at 0.508 fails, 11 at 0.902 fails.

With the old gate retired, natural data was finally searched: 0.0000
against 0.0039 chance. The old verdict was right, its reasoning was not.

Also closes route D by measurement. rho_IT ~ sqrt(4 log N / N) rises as N
falls, and at N = 16 through 96 the deepest state a strong searcher
reaches is deeper than the truth in 3/3 replicates at every size.

Free gains: eigenvalue-weighted projection over a wide basis with
128-dim text vectors takes the correlation 0.656 -> 0.716 and shared
width 10 -> 16. Hubness refuted as an inflation hypothesis.

Moving the per-image segmentation eigendecomposition onto the GPU cut
batch time from 130s to 1.9s.

Co-Authored-By: Claude <noreply@anthropic.com>

6 files changed, 221 insertions, 0 deletions

diff --git a/artifacts/synth_v1/omit_colour.json b/artifacts/synth_v1/omit_colour.json
new file mode 100644
index 0000000..61f71bc
--- /dev/null
+++ b/artifacts/synth_v1/omit_colour.json
@@ -0,0 +1,34 @@
+{
+  "data_dir": "artifacts/synth_v1",
+  "split": "test",
+  "samples": 256,
+  "colour_classes": 10,
+  "text_families": {
+    "count_words": [
+      "a",
+      "three",
+      "four",
+      "two"
+    ],
+    "colour_words": [
+      "red",
+      "orange",
+      "yellow",
+      "green",
+      "blue",
+      "purple",
+      "pink",
+      "white",
+      "gray",
+      "brown"
+    ],
+    "size_words": [
+      [
+        "small",
+        "large"
+      ]
+    ]
+  },
+  "field_correlation_at_truth": 0.7356797393477761,
+  "note": "The dictionary is derived from disjoint corpora; the correlation is a diagnostic computed with hidden pairs and never used by the pipeline."
+}
\ No newline at end of file
diff --git a/artifacts/synth_v1/omit_count.json b/artifacts/synth_v1/omit_count.json
new file mode 100644
index 0000000..7a40369
--- /dev/null
+++ b/artifacts/synth_v1/omit_count.json
@@ -0,0 +1,34 @@
+{
+  "data_dir": "artifacts/synth_v1",
+  "split": "test",
+  "samples": 256,
+  "colour_classes": 10,
+  "text_families": {
+    "count_words": [
+      "a",
+      "three",
+      "four",
+      "two"
+    ],
+    "colour_words": [
+      "red",
+      "orange",
+      "yellow",
+      "green",
+      "blue",
+      "purple",
+      "pink",
+      "white",
+      "gray",
+      "brown"
+    ],
+    "size_words": [
+      [
+        "small",
+        "large"
+      ]
+    ]
+  },
+  "field_correlation_at_truth": 0.678442918934101,
+  "note": "The dictionary is derived from disjoint corpora; the correlation is a diagnostic computed with hidden pairs and never used by the pipeline."
+}
\ No newline at end of file
diff --git a/artifacts/synth_v1/omit_none.json b/artifacts/synth_v1/omit_none.json
new file mode 100644
index 0000000..7ff06f9
--- /dev/null
+++ b/artifacts/synth_v1/omit_none.json
@@ -0,0 +1,34 @@
+{
+  "data_dir": "artifacts/synth_v1",
+  "split": "test",
+  "samples": 256,
+  "colour_classes": 10,
+  "text_families": {
+    "count_words": [
+      "a",
+      "three",
+      "four",
+      "two"
+    ],
+    "colour_words": [
+      "red",
+      "orange",
+      "yellow",
+      "green",
+      "blue",
+      "purple",
+      "pink",
+      "white",
+      "gray",
+      "brown"
+    ],
+    "size_words": [
+      [
+        "small",
+        "large"
+      ]
+    ]
+  },
+  "field_correlation_at_truth": 0.9284809883663888,
+  "note": "The dictionary is derived from disjoint corpora; the correlation is a diagnostic computed with hidden pairs and never used by the pipeline."
+}
\ No newline at end of file
diff --git a/artifacts/synth_v1/omit_size.json b/artifacts/synth_v1/omit_size.json
new file mode 100644
index 0000000..d81aa00
--- /dev/null
+++ b/artifacts/synth_v1/omit_size.json
@@ -0,0 +1,34 @@
+{
+  "data_dir": "artifacts/synth_v1",
+  "split": "test",
+  "samples": 256,
+  "colour_classes": 10,
+  "text_families": {
+    "count_words": [
+      "a",
+      "three",
+      "four",
+      "two"
+    ],
+    "colour_words": [
+      "red",
+      "orange",
+      "yellow",
+      "green",
+      "blue",
+      "purple",
+      "pink",
+      "white",
+      "gray",
+      "brown"
+    ],
+    "size_words": [
+      [
+        "small",
+        "large"
+      ]
+    ]
+  },
+  "field_correlation_at_truth": 0.8301825436675783,
+  "note": "The dictionary is derived from disjoint corpora; the correlation is a diagnostic computed with hidden pairs and never used by the pipeline."
+}
\ No newline at end of file
diff --git a/artifacts/synth_v1/omit_size_count.json b/artifacts/synth_v1/omit_size_count.json
new file mode 100644
index 0000000..2eca4dc
--- /dev/null
+++ b/artifacts/synth_v1/omit_size_count.json
@@ -0,0 +1,34 @@
+{
+  "data_dir": "artifacts/synth_v1",
+  "split": "test",
+  "samples": 256,
+  "colour_classes": 10,
+  "text_families": {
+    "count_words": [
+      "a",
+      "three",
+      "four",
+      "two"
+    ],
+    "colour_words": [
+      "red",
+      "orange",
+      "yellow",
+      "green",
+      "blue",
+      "purple",
+      "pink",
+      "white",
+      "gray",
+      "brown"
+    ],
+    "size_words": [
+      [
+        "small",
+        "large"
+      ]
+    ]
+  },
+  "field_correlation_at_truth": 0.5442660429860929,
+  "note": "The dictionary is derived from disjoint corpora; the correlation is a diagnostic computed with hidden pairs and never used by the pipeline."
+}
\ No newline at end of file
diff --git a/artifacts/synth_v1/rank_causal.json b/artifacts/synth_v1/rank_causal.json
new file mode 100644
index 0000000..222e469
--- /dev/null
+++ b/artifacts/synth_v1/rank_causal.json
@@ -0,0 +1,51 @@
+{
+  "protocol": "Both fields truncated to the same rank, then matched blind from a spectral start with exact refinement. Hidden pairing generated per trial and read only for scoring.",
+  "label": "synthetic-watershed",
+  "size": 256,
+  "chance": 0.00390625,
+  "rows": [
+    {
+      "rank": 4,
+      "correlation": 0.9016677754313268,
+      "recovery": 0.0625,
+      "visual_effective_rank": 44.86571084196529
+    },
+    {
+      "rank": 8,
+      "correlation": 0.9058447537972105,
+      "recovery": 0.12890625,
+      "visual_effective_rank": 71.3420544398913
+    },
+    {
+      "rank": 16,
+      "correlation": 0.9281446653770646,
+      "recovery": 0.9557291666666666,
+      "visual_effective_rank": 95.72603497323556
+    },
+    {
+      "rank": 32,
+      "correlation": 0.9282073730691501,
+      "recovery": 0.9375,
+      "visual_effective_rank": 109.47942916160699
+    },
+    {
+      "rank": 64,
+      "correlation": 0.9284208632987705,
+      "recovery": 0.9583333333333334,
+      "visual_effective_rank": 106.31167521180129
+    },
+    {
+      "rank": 128,
+      "correlation": 0.9284791170404748,
+      "recovery": 0.9609375,
+      "visual_effective_rank": 106.04029186749484
+    },
+    {
+      "rank": 256,
+      "correlation": 0.928480988366389,
+      "recovery": 0.9583333333333334,
+      "visual_effective_rank": 106.03700034539153
+    }
+  ],
+  "reading": "Recovery collapsing at low rank while the correlation stays above the recovery threshold shows the headline correlation is not sufficient and the shared spectrum's width is the binding constraint."
+}
\ No newline at end of file