Visual Genome is capped: part correspondence buys 0.023

Each VG region box is index-aligned with its own description, so the crop
can be encoded and paired with the phrase describing it -- part
correspondence by annotation, which is cross-modal supervision a deployed
system would not have. The cross-modal anchor bound goes 0.336 -> 0.359.

So the two encoders do not covary at the part level either, and no
segmentation, aggregation, kernel or backbone repairs that. The 0.805
within-text ceiling was never the cross-modal ceiling; for this encoder
pair on this corpus the cross-modal ceiling is about 0.36 against the 0.9
recovery needs.

Six representational interventions this session were chosen without a
ceiling in view -- the same error being made on the matching side at the
same time. What replaces them is a three-number screening protocol that
runs before any pipeline is built.

Co-Authored-By: Claude <noreply@anthropic.com>

4 files changed, 293 insertions, 0 deletions

diff --git a/NATURAL_RESULTS.md b/NATURAL_RESULTS.md
index e0ea4b7..502acb3 100644
--- a/NATURAL_RESULTS.md
+++ b/NATURAL_RESULTS.md
@@ -135,3 +135,53 @@ observation is necessary now has a natural-data counterpart to test.
 - `artifacts/vg_5k/natural_families{,_dist}.json`
 - `artifacts/vg_5k/natural_objects{,_16,_oracle}.pt`
 - `artifacts/vg_5k/natural_fields{,_16}.pt`
+
+---
+
+# The corpus is capped, and the part correspondence was never the problem
+
+*Appended 2026-08-01.*
+
+Six representational interventions were tried on Visual Genome across one
+session — a wider weighted projection, larger text vectors, structured phrase
+encoding, more segments, annotated boxes instead of segmentation, third-order
+moments in the set kernel, and a backbone at twice the depth. They moved the
+field correlation from 0.656 to 0.731 and the cross-modal anchor bound from
+0.195 to 0.336. None was chosen against a ceiling, because no ceiling had been
+measured. That was the same error being made on the matching side at the same
+time: **optimising without a bound.**
+
+Three measurements now bound it, and they disagree with where the effort went.
+
+| Measurement | Value |
+|---|---|
+| Vision identifies its own scenes (split-half) | **0.994** |
+| Text identifies its own scenes (split-half, IDF bag-of-words) | **0.805** |
+| Text identifies its own scenes (our encoding pipeline) | 0.491 |
+| Cross-modal, unsupervised parts | 0.336 |
+| **Cross-modal, annotator-supplied part correspondence** | **0.359** |
+
+Each region box in Visual Genome is index-aligned with its own description, so
+the crop inside the box can be encoded and paired with the phrase that
+describes it — putting the two modalities' parts in correspondence by
+annotation, which is cross-modal supervision a deployed system would never
+have. **It buys 0.023.**
+
+That settles the question the pipeline work could not. The two encoders do not
+covary at the part level either: a DINOv2 descriptor of a region and an English
+phrase describing the same region do not agree about what makes two regions
+alike, and no segmentation, aggregation, kernel or backbone repairs that. The
+0.805 within-text ceiling was never the cross-modal ceiling; the cross-modal
+ceiling for this encoder pair on this corpus is about **0.36**, against the 0.9
+that recovery needs.
+
+**Visual Genome is closed.** Not because the method fails but because this
+pairing of encoder and corpus does not carry the agreement the method consumes,
+and that is now a measurement rather than a suspicion.
+
+What it leaves is a screening protocol rather than a repair list. Three numbers,
+all cheap and all computable before any pipeline is built: each modality's
+split-half self-identification, the cross-modal anchor bound, and — where the
+corpus offers aligned parts — the same bound with part correspondence given.
+A corpus whose cross-modal bound sits near its within-modality ceilings is worth
+building on; Visual Genome's sits at 45% of the lower of them.
diff --git a/artifacts/vg_5k/region_oracle.json b/artifacts/vg_5k/region_oracle.json
new file mode 100644
index 0000000..969266a
--- /dev/null
+++ b/artifacts/vg_5k/region_oracle.json
@@ -0,0 +1,10 @@
+{
+  "protocol": "DIAGNOSTIC, NOT A METHOD. Each annotated region box is encoded as a crop and paired with its own description, so the two modalities carry the same parts in the same order. That box-to-phrase link is cross-modal supervision a deployed system would not have.",
+  "scenes": 256,
+  "mean_parts": 15.95703125,
+  "correlation": 0.6646459453410474,
+  "anchor_bound_with_part_oracle": 0.359375,
+  "anchor_bound_unsupervised_reference": 0.336,
+  "within_modality_ceiling_text": 0.805,
+  "reading": "A large jump means part correspondence is the whole problem and is an engineering target. Staying near 0.336 means the two encoders do not covary at the part level either, and no aggregation rescues this corpus."
+}
\ No newline at end of file
diff --git a/logs/region_oracle.log b/logs/region_oracle.log
new file mode 100644
index 0000000..bbfc070
--- /dev/null
+++ b/logs/region_oracle.log
@@ -0,0 +1,17 @@
+1156 scenes
+Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+`torch_dtype` is deprecated! Use `dtype` instead!
+
crop:   0%|          | 0/1156 [00:00<?, ?it/s]
crop:   1%|          | 6/1156 [00:00<00:21, 52.59it/s]
crop:   1%|          | 13/1156 [00:00<00:18, 61.61it/s]
crop:   2%|▏         | 22/1156 [00:00<00:15, 73.49it/s]
crop:   3%|▎         | 33/1156 [00:00<00:12, 87.45it/s]
crop:   4%|▍         | 44/1156 [00:00<00:12, 92.29it/s]
crop:   5%|▍         | 54/1156 [00:00<00:12, 86.26it/s]
crop:   6%|▌         | 65/1156 [00:00<00:11, 93.11it/s]
crop:   6%|▋         | 75/1156 [00:00<00:11, 94.49it/s]
crop:   7%|▋         | 85/1156 [00:00<00:11, 95.47it/s]
crop:   8%|▊         | 97/1156 [00:01<00:10, 99.76it/s]
crop:   9%|▉         | 108/1156 [00:01<00:11, 91.48it/s]
crop:  10%|█         | 118/1156 [00:01<00:13, 76.94it/s]
crop:  11%|█         | 127/1156 [00:01<00:12, 79.27it/s]
crop:  12%|█▏        | 136/1156 [00:01<00:13, 76.35it/s]
crop:  13%|█▎        | 146/1156 [00:01<00:12, 80.62it/s]
crop:  13%|█▎        | 155/1156 [00:01<00:12, 79.34it/s]
crop:  14%|█▍        | 164/1156 [00:01<00:12, 81.17it/s]
crop:  15%|█▍        | 173/1156 [00:02<00:12, 78.93it/s]
crop:  16%|█▌        | 183/1156 [00:02<00:11, 82.55it/s]
crop:  17%|█▋        | 193/1156 [00:02<00:11, 86.60it/s]
crop:  18%|█▊        | 203/1156 [00:02<00:10, 90.00it/s]
crop:  18%|█▊        | 213/1156 [00:02<00:11, 81.01it/s]
crop:  19%|█▉        | 222/1156 [00:02<00:14, 64.13it/s]
crop:  20%|██        | 235/1156 [00:02<00:11, 76.90it/s]
crop:  21%|██        | 244/1156 [00:03<00:12, 71.64it/s]
crop:  22%|██▏       | 255/1156 [00:03<00:11, 79.57it/s]
crop:  23%|██▎       | 268/1156 [00:03<00:09, 90.31it/s]
crop:  24%|██▍       | 278/1156 [00:03<00:10, 85.72it/s]
crop:  25%|██▌       | 290/1156 [00:03<00:09, 92.54it/s]
crop:  26%|██▌       | 300/1156 [00:03<00:09, 92.14it/s]
crop:  27%|██▋       | 310/1156 [00:03<00:11, 73.47it/s]
crop:  28%|██▊       | 321/1156 [00:03<00:10, 80.27it/s]
crop:  29%|██▊       | 331/1156 [00:04<00:09, 83.78it/s]
crop:  30%|██▉       | 342/1156 [00:04<00:09, 88.55it/s]
crop:  30%|███       | 352/1156 [00:04<00:09, 84.63it/s]
crop:  31%|███       | 361/1156 [00:04<00:09, 85.07it/s]
crop:  32%|███▏      | 374/1156 [00:04<00:08, 95.72it/s]
crop:  33%|███▎      | 386/1156 [00:04<00:07, 100.50it/s]
crop:  34%|███▍      | 398/1156 [00:04<00:07, 99.44it/s] 
crop:  35%|███▌      | 409/1156 [00:04<00:07, 100.28it/s]
crop:  36%|███▋      | 420/1156 [00:04<00:08, 90.23it/s] 
crop:  37%|███▋      | 431/1156 [00:05<00:07, 92.98it/s]
crop:  38%|███▊      | 441/1156 [00:05<00:07, 94.24it/s]
crop:  39%|███▉      | 451/1156 [00:05<00:08, 82.33it/s]
crop:  40%|███▉      | 462/1156 [00:05<00:07, 89.22it/s]
crop:  41%|████      | 474/1156 [00:05<00:07, 96.69it/s]
crop:  42%|████▏     | 486/1156 [00:05<00:06, 100.19it/s]
crop:  43%|████▎     | 499/1156 [00:05<00:06, 107.22it/s]
crop:  44%|████▍     | 513/1156 [00:05<00:05, 115.71it/s]
crop:  46%|████▌     | 527/1156 [00:05<00:05, 120.71it/s]
crop:  47%|████▋     | 540/1156 [00:06<00:05, 119.84it/s]
crop:  48%|████▊     | 554/1156 [00:06<00:04, 125.33it/s]
crop:  49%|████▉     | 569/1156 [00:06<00:04, 130.87it/s]
crop:  50%|█████     | 583/1156 [00:06<00:04, 130.27it/s]
crop:  52%|█████▏    | 597/1156 [00:06<00:04, 128.32it/s]
crop:  53%|█████▎    | 610/1156 [00:06<00:04, 119.72it/s]
crop:  54%|█████▍    | 623/1156 [00:06<00:04, 114.85it/s]
crop:  55%|█████▍    | 635/1156 [00:06<00:05, 90.86it/s] 
crop:  56%|█████▌    | 645/1156 [00:07<00:05, 91.48it/s]
crop:  57%|█████▋    | 655/1156 [00:07<00:05, 90.08it/s]
crop:  58%|█████▊    | 666/1156 [00:07<00:05, 92.71it/s]
crop:  58%|█████▊    | 676/1156 [00:07<00:05, 90.52it/s]
crop:  59%|█████▉    | 686/1156 [00:07<00:05, 90.78it/s]
crop:  60%|██████    | 696/1156 [00:07<00:04, 93.06it/s]
crop:  61%|██████    | 706/1156 [00:07<00:07, 60.77it/s]
crop:  62%|██████▏   | 717/1156 [00:08<00:06, 69.75it/s]
crop:  63%|██████▎   | 726/1156 [00:08<00:05, 73.95it/s]
crop:  64%|██████▍   | 739/1156 [00:08<00:04, 86.94it/s]
crop:  65%|██████▍   | 749/1156 [00:08<00:04, 87.75it/s]
crop:  66%|██████▌   | 761/1156 [00:08<00:04, 96.02it/s]
crop:  67%|██████▋   | 775/1156 [00:08<00:03, 106.20it/s]
crop:  68%|██████▊   | 787/1156 [00:08<00:03, 107.75it/s]
crop:  69%|██████▉   | 799/1156 [00:08<00:04, 73.42it/s] 
crop:  70%|██████▉   | 809/1156 [00:09<00:04, 76.55it/s]
crop:  71%|███████   | 819/1156 [00:09<00:04, 77.63it/s]
crop:  72%|███████▏  | 828/1156 [00:09<00:04, 79.58it/s]
crop:  72%|███████▏  | 837/1156 [00:09<00:04, 74.42it/s]
crop:  73%|███████▎  | 848/1156 [00:09<00:03, 82.55it/s]
crop:  74%|███████▍  | 857/1156 [00:09<00:03, 84.45it/s]
crop:  75%|███████▍  | 866/1156 [00:09<00:03, 83.28it/s]
crop:  76%|███████▌  | 875/1156 [00:10<00:05, 55.71it/s]
crop:  77%|███████▋  | 886/1156 [00:10<00:04, 66.35it/s]
crop:  78%|███████▊  | 898/1156 [00:10<00:03, 78.22it/s]
crop:  79%|███████▊  | 908/1156 [00:10<00:03, 76.92it/s]
crop:  79%|███████▉  | 918/1156 [00:10<00:02, 79.88it/s]
crop:  80%|████████  | 927/1156 [00:10<00:02, 80.24it/s]
crop:  81%|████████  | 938/1156 [00:10<00:02, 86.52it/s]
crop:  82%|████████▏ | 949/1156 [00:10<00:02, 90.62it/s]
crop:  83%|████████▎ | 959/1156 [00:10<00:02, 91.60it/s]
crop:  84%|████████▍ | 969/1156 [00:11<00:02, 80.21it/s]
crop:  85%|████████▍ | 981/1156 [00:11<00:01, 90.10it/s]
crop:  86%|████████▌ | 991/1156 [00:11<00:01, 89.72it/s]
crop:  87%|████████▋ | 1001/1156 [00:11<00:01, 83.88it/s]
crop:  87%|████████▋ | 1011/1156 [00:11<00:01, 87.21it/s]
crop:  88%|████████▊ | 1021/1156 [00:11<00:01, 90.41it/s]
crop:  89%|████████▉ | 1031/1156 [00:11<00:01, 92.76it/s]
crop:  90%|█████████ | 1041/1156 [00:11<00:01, 91.67it/s]
crop:  91%|█████████ | 1052/1156 [00:11<00:01, 95.66it/s]
crop:  92%|█████████▏| 1064/1156 [00:12<00:00, 102.51it/s]
crop:  93%|█████████▎| 1075/1156 [00:12<00:00, 102.23it/s]
crop:  94%|█████████▍| 1087/1156 [00:12<00:00, 107.02it/s]
crop:  95%|█████████▌| 1101/1156 [00:12<00:00, 113.92it/s]
crop:  96%|█████████▋| 1113/1156 [00:12<00:00, 111.18it/s]
crop:  97%|█████████▋| 1125/1156 [00:12<00:00, 113.18it/s]
crop:  98%|█████████▊| 1138/1156 [00:12<00:00, 116.78it/s]
crop:  99%|█████████▉| 1150/1156 [00:12<00:00, 104.42it/s]
crop: 100%|██████████| 1156/1156 [00:12<00:00, 89.85it/s] 
+
encode:   0%|          | 0/193 [00:00<?, ?it/s]
encode:   1%|          | 1/193 [00:00<01:26,  2.23it/s]
encode:   1%|          | 2/193 [00:00<00:47,  4.03it/s]
encode:   2%|▏         | 3/193 [00:00<00:34,  5.45it/s]
encode:   3%|▎         | 5/193 [00:00<00:27,  6.72it/s]
encode:   4%|▎         | 7/193 [00:01<00:23,  7.87it/s]
encode:   4%|▍         | 8/193 [00:01<00:27,  6.80it/s]
encode:   5%|▌         | 10/193 [00:01<00:23,  7.70it/s]
encode:   6%|▌         | 11/193 [00:01<00:22,  8.05it/s]
encode:   7%|▋         | 13/193 [00:01<00:20,  8.62it/s]
encode:   7%|▋         | 14/193 [00:01<00:20,  8.83it/s]
encode:   8%|▊         | 15/193 [00:02<00:20,  8.83it/s]
encode:   9%|▉         | 17/193 [00:02<00:18,  9.31it/s]
encode:   9%|▉         | 18/193 [00:02<00:19,  8.90it/s]
encode:  10%|▉         | 19/193 [00:02<00:19,  9.11it/s]
encode:  11%|█         | 21/193 [00:02<00:18,  9.47it/s]
encode:  11%|█▏        | 22/193 [00:02<00:17,  9.58it/s]
encode:  12%|█▏        | 23/193 [00:02<00:18,  9.19it/s]
encode:  12%|█▏        | 24/193 [00:03<00:18,  9.27it/s]
encode:  13%|█▎        | 26/193 [00:03<00:17,  9.65it/s]
encode:  14%|█▍        | 27/193 [00:03<00:18,  8.98it/s]
encode:  15%|█▍        | 28/193 [00:03<00:19,  8.45it/s]
encode:  15%|█▌        | 29/193 [00:03<00:18,  8.63it/s]
encode:  16%|█▌        | 30/193 [00:03<00:19,  8.29it/s]
encode:  16%|█▌        | 31/193 [00:03<00:18,  8.62it/s]
encode:  17%|█▋        | 33/193 [00:04<00:17,  9.21it/s]
encode:  18%|█▊        | 34/193 [00:04<00:17,  9.05it/s]
encode:  18%|█▊        | 35/193 [00:04<00:21,  7.50it/s]
encode:  19%|█▉        | 37/193 [00:04<00:18,  8.45it/s]
encode:  20%|█▉        | 38/193 [00:04<00:17,  8.73it/s]
encode:  20%|██        | 39/193 [00:04<00:17,  8.56it/s]
encode:  21%|██        | 40/193 [00:04<00:18,  8.40it/s]
encode:  21%|██        | 41/193 [00:05<00:17,  8.48it/s]
encode:  22%|██▏       | 42/193 [00:05<00:17,  8.80it/s]
encode:  22%|██▏       | 43/193 [00:05<00:19,  7.67it/s]
encode:  23%|██▎       | 45/193 [00:05<00:17,  8.55it/s]
encode:  24%|██▍       | 47/193 [00:05<00:16,  8.88it/s]
encode:  25%|██▍       | 48/193 [00:05<00:15,  9.09it/s]
encode:  26%|██▌       | 50/193 [00:06<00:17,  8.29it/s]
encode:  26%|██▋       | 51/193 [00:06<00:16,  8.53it/s]
encode:  27%|██▋       | 53/193 [00:06<00:15,  8.79it/s]
encode:  28%|██▊       | 54/193 [00:06<00:18,  7.64it/s]
encode:  29%|██▉       | 56/193 [00:06<00:20,  6.75it/s]
encode:  30%|██▉       | 57/193 [00:07<00:20,  6.71it/s]
encode:  30%|███       | 58/193 [00:07<00:18,  7.25it/s]
encode:  31%|███       | 60/193 [00:07<00:16,  8.13it/s]
encode:  32%|███▏      | 62/193 [00:07<00:14,  8.83it/s]
encode:  33%|███▎      | 64/193 [00:07<00:13,  9.31it/s]
encode:  34%|███▎      | 65/193 [00:07<00:14,  8.87it/s]
encode:  34%|███▍      | 66/193 [00:08<00:17,  7.19it/s]
encode:  35%|███▍      | 67/193 [00:08<00:17,  7.21it/s]
encode:  35%|███▌      | 68/193 [00:08<00:19,  6.46it/s]
encode:  36%|███▋      | 70/193 [00:08<00:16,  7.61it/s]
encode:  37%|███▋      | 71/193 [00:08<00:15,  7.95it/s]
encode:  38%|███▊      | 73/193 [00:09<00:16,  7.27it/s]
encode:  38%|███▊      | 74/193 [00:09<00:18,  6.32it/s]
encode:  39%|███▉      | 75/193 [00:09<00:20,  5.80it/s]
encode:  40%|███▉      | 77/193 [00:09<00:16,  6.96it/s]
encode:  41%|████      | 79/193 [00:10<00:16,  6.75it/s]
encode:  42%|████▏     | 81/193 [00:10<00:14,  7.53it/s]
encode:  43%|████▎     | 83/193 [00:10<00:13,  8.27it/s]
encode:  44%|████▎     | 84/193 [00:10<00:12,  8.52it/s]
encode:  44%|████▍     | 85/193 [00:10<00:12,  8.65it/s]
encode:  45%|████▌     | 87/193 [00:10<00:12,  8.65it/s]
encode:  46%|████▌     | 89/193 [00:11<00:11,  8.99it/s]
encode:  47%|████▋     | 90/193 [00:11<00:12,  8.24it/s]
encode:  47%|████▋     | 91/193 [00:11<00:12,  8.07it/s]
encode:  48%|████▊     | 92/193 [00:11<00:12,  8.21it/s]
encode:  48%|████▊     | 93/193 [00:11<00:11,  8.54it/s]
encode:  49%|████▉     | 95/193 [00:11<00:10,  8.97it/s]
encode:  50%|████▉     | 96/193 [00:12<00:13,  6.96it/s]
encode:  51%|█████     | 98/193 [00:12<00:11,  7.97it/s]
encode:  51%|█████▏    | 99/193 [00:12<00:11,  8.01it/s]
encode:  52%|█████▏    | 100/193 [00:12<00:11,  8.41it/s]
encode:  52%|█████▏    | 101/193 [00:12<00:10,  8.66it/s]
encode:  53%|█████▎    | 103/193 [00:12<00:09,  9.10it/s]
encode:  54%|█████▍    | 104/193 [00:12<00:09,  9.12it/s]
encode:  54%|█████▍    | 105/193 [00:13<00:09,  9.18it/s]
encode:  55%|█████▌    | 107/193 [00:13<00:08,  9.63it/s]
encode:  56%|█████▋    | 109/193 [00:13<00:09,  8.42it/s]
encode:  57%|█████▋    | 110/193 [00:13<00:09,  8.35it/s]
encode:  58%|█████▊    | 111/193 [00:13<00:09,  8.57it/s]
encode:  58%|█████▊    | 112/193 [00:14<00:13,  5.91it/s]
encode:  59%|█████▉    | 114/193 [00:14<00:11,  7.16it/s]
encode:  60%|█████▉    | 115/193 [00:14<00:14,  5.23it/s]
encode:  60%|██████    | 116/193 [00:14<00:13,  5.86it/s]
encode:  61%|██████    | 117/193 [00:14<00:11,  6.50it/s]
encode:  61%|██████    | 118/193 [00:14<00:10,  6.96it/s]
encode:  62%|██████▏   | 119/193 [00:15<00:11,  6.53it/s]
encode:  62%|██████▏   | 120/193 [00:15<00:10,  7.21it/s]
encode:  63%|██████▎   | 121/193 [00:15<00:09,  7.83it/s]
encode:  64%|██████▎   | 123/193 [00:15<00:08,  8.18it/s]
encode:  65%|██████▍   | 125/193 [00:15<00:07,  8.84it/s]
encode:  65%|██████▌   | 126/193 [00:15<00:08,  8.34it/s]
encode:  66%|██████▌   | 127/193 [00:16<00:07,  8.64it/s]
encode:  66%|██████▋   | 128/193 [00:16<00:07,  8.82it/s]
encode:  67%|██████▋   | 129/193 [00:16<00:07,  8.46it/s]
encode:  68%|██████▊   | 131/193 [00:16<00:06,  9.14it/s]
encode:  69%|██████▉   | 133/193 [00:16<00:06,  9.46it/s]
encode:  69%|██████▉   | 134/193 [00:16<00:06,  9.28it/s]
encode:  70%|███████   | 136/193 [00:17<00:06,  8.94it/s]
encode:  71%|███████   | 137/193 [00:17<00:06,  9.14it/s]
encode:  72%|███████▏  | 138/193 [00:17<00:06,  9.08it/s]
encode:  72%|███████▏  | 139/193 [00:17<00:05,  9.29it/s]
encode:  73%|███████▎  | 140/193 [00:17<00:05,  9.03it/s]
encode:  74%|███████▎  | 142/193 [00:17<00:05,  9.53it/s]
encode:  74%|███████▍  | 143/193 [00:17<00:05,  9.24it/s]
encode:  75%|███████▍  | 144/193 [00:17<00:05,  9.34it/s]
encode:  75%|███████▌  | 145/193 [00:17<00:05,  8.78it/s]
encode:  76%|███████▌  | 146/193 [00:18<00:05,  9.07it/s]
encode:  76%|███████▌  | 147/193 [00:18<00:05,  9.12it/s]
encode:  77%|███████▋  | 148/193 [00:18<00:04,  9.09it/s]
encode:  77%|███████▋  | 149/193 [00:18<00:04,  9.30it/s]
encode:  78%|███████▊  | 151/193 [00:18<00:04,  9.52it/s]
encode:  79%|███████▉  | 153/193 [00:18<00:04,  9.78it/s]
encode:  80%|███████▉  | 154/193 [00:18<00:04,  8.98it/s]
encode:  80%|████████  | 155/193 [00:19<00:04,  9.16it/s]
encode:  81%|████████  | 156/193 [00:19<00:03,  9.35it/s]
encode:  81%|████████▏ | 157/193 [00:19<00:04,  8.88it/s]
encode:  82%|████████▏ | 159/193 [00:19<00:03,  9.13it/s]
encode:  83%|████████▎ | 161/193 [00:19<00:03,  9.52it/s]
encode:  84%|████████▍ | 162/193 [00:19<00:03,  9.60it/s]
encode:  84%|████████▍ | 163/193 [00:19<00:03,  9.45it/s]
encode:  85%|████████▍ | 164/193 [00:20<00:03,  8.70it/s]
encode:  85%|████████▌ | 165/193 [00:20<00:03,  9.01it/s]
encode:  87%|████████▋ | 167/193 [00:20<00:02,  9.37it/s]
encode:  88%|████████▊ | 169/193 [00:20<00:02,  9.67it/s]
encode:  89%|████████▊ | 171/193 [00:20<00:02,  9.73it/s]
encode:  90%|████████▉ | 173/193 [00:21<00:02,  8.45it/s]
encode:  91%|█████████ | 175/193 [00:21<00:02,  8.89it/s]
encode:  91%|█████████ | 176/193 [00:21<00:01,  9.06it/s]
encode:  92%|█████████▏| 177/193 [00:21<00:01,  8.21it/s]
encode:  92%|█████████▏| 178/193 [00:21<00:01,  8.50it/s]
encode:  93%|█████████▎| 179/193 [00:21<00:01,  8.75it/s]
encode:  93%|█████████▎| 180/193 [00:21<00:01,  8.85it/s]
encode:  94%|█████████▍| 181/193 [00:21<00:01,  7.90it/s]
encode:  94%|█████████▍| 182/193 [00:22<00:01,  8.24it/s]
encode:  95%|█████████▍| 183/193 [00:22<00:01,  8.60it/s]
encode:  96%|█████████▌| 185/193 [00:22<00:00,  8.37it/s]
encode:  96%|█████████▋| 186/193 [00:22<00:00,  8.62it/s]
encode:  97%|█████████▋| 187/193 [00:22<00:00,  8.81it/s]
encode:  98%|█████████▊| 189/193 [00:22<00:00,  8.71it/s]
encode:  98%|█████████▊| 190/193 [00:23<00:00,  8.16it/s]
encode:  99%|█████████▉| 191/193 [00:23<00:00,  8.47it/s]
encode:  99%|█████████▉| 192/193 [00:23<00:00,  8.62it/s]
encode: 100%|██████████| 193/193 [00:23<00:00,  8.27it/s]
+18496 region crops encoded
+256 evaluated, 900 for fitting
+{
+  "protocol": "DIAGNOSTIC, NOT A METHOD. Each annotated region box is encoded as a crop and paired with its own description, so the two modalities carry the same parts in the same order. That box-to-phrase link is cross-modal supervision a deployed system would not have.",
+  "scenes": 256,
+  "mean_parts": 15.95703125,
+  "correlation": 0.6646459453410474,
+  "anchor_bound_with_part_oracle": 0.359375,
+  "anchor_bound_unsupervised_reference": 0.336,
+  "within_modality_ceiling_text": 0.805,
+  "reading": "A large jump means part correspondence is the whole problem and is an engineering target. Staying near 0.336 means the two encoders do not covary at the part level either, and no aggregation rescues this corpus."
+}
diff --git a/worldalign/region_oracle.py b/worldalign/region_oracle.py
new file mode 100644
index 0000000..61054ee
--- /dev/null
+++ b/worldalign/region_oracle.py
@@ -0,0 +1,216 @@
+"""Is Visual Genome capped, or is our part correspondence the problem?
+
+Visual Genome's cross-modal anchor bound is 0.336 against a within-modality
+ceiling of 0.805 for text and 0.994 for vision. Both sides identify their own
+scenes; they disagree about which scenes resemble each other. That leaves two
+very different explanations and they call for opposite responses.
+
+Either the two encoders would agree if only their parts were put in
+correspondence -- our spectral segments and the annotators' phrases carve the
+image differently, and the aggregation has to guess which goes with which --
+or they would not, because a DINOv2 patch descriptor and a written phrase
+simply do not covary over what makes two regions alike.
+
+Visual Genome settles it, because each region box is index-aligned with its own
+description. Encode the crop inside each box and the phrase describing it, and
+the parts are in correspondence **by annotation**. If the cross-modal bound
+jumps, part correspondence is the whole problem and it is an engineering target.
+If it stays near 0.336, the encoders do not agree at the part level either, and
+no aggregation will rescue this corpus.
+
+DIAGNOSTIC ONLY. Box-to-phrase alignment is cross-modal supervision that a
+deployed system would not have. This measures a ceiling, it is not a method.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import re
+from pathlib import Path
+
+import numpy as np
+import torch
+import torch.nn.functional as F
+from PIL import Image
+from scipy.optimize import linear_sum_assignment
+from tqdm import tqdm
+from transformers import AutoImageProcessor, AutoModel
+
+from .common import seed_everything, write_json
+from .natural_families import load_phrases
+from .natural_pipeline import content_directions, word_vectors
+from .synth_cc_battery import moment_field
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--vg-dir", default="artifacts/vg_5k")
+    parser.add_argument("--image-cache", default="/tmp/yurenh2-worldalign-vg-images")
+    parser.add_argument("--model", default="facebook/dinov2-base")
+    parser.add_argument("--crop-size", type=int, default=126)
+    parser.add_argument("--samples", type=int, default=256)
+    parser.add_argument("--fit-scenes", type=int, default=900)
+    parser.add_argument("--word-vectors", type=int, default=128)
+    parser.add_argument("--min-count", type=int, default=60)
+    parser.add_argument("--max-vocabulary", type=int, default=600)
+    parser.add_argument("--context-window", type=int, default=2)
+    parser.add_argument("--keep", type=int, default=64)
+    parser.add_argument("--shrinkage", type=float, default=0.05)
+    parser.add_argument("--weight-power", type=float, default=0.5)
+    parser.add_argument("--batch-size", type=int, default=64)
+    parser.add_argument("--device", default="cuda:3")
+    parser.add_argument("--seed", type=int, default=0)
+    parser.add_argument("--output", default="artifacts/vg_5k/region_oracle.json")
+    return parser.parse_args()
+
+
+def standardise(matrix: np.ndarray) -> np.ndarray:
+    mask = ~np.eye(len(matrix), dtype=bool)
+    values = matrix[mask]
+    out = (matrix - values.mean()) / values.std()
+    np.fill_diagonal(out, 0.0)
+    return out
+
+
+def anchor_bound(visual: np.ndarray, text: np.ndarray, repeats: int = 5) -> float:
+    size = len(visual)
+    scores = []
+    for repeat in range(repeats):
+        generator = np.random.default_rng(repeat)
+        shuffle = generator.permutation(size)
+        half = size // 2
+        anchors, probe = shuffle[:half], shuffle[half:]
+        order = generator.permutation(len(probe))
+
+        def profile(field, rows):
+            block = field[np.ix_(rows, anchors)]
+            block = block - block.mean(1, keepdims=True)
+            return block / block.std(1, keepdims=True).clip(1e-9)
+
+        similarity = profile(visual, probe) @ profile(text, probe[order]).T / half
+        _, columns = linear_sum_assignment(-similarity)
+        scores.append(float((order[columns] == np.arange(len(probe))).mean()))
+    return float(np.mean(scores))
+
+
+@torch.inference_mode()
+def main() -> None:
+    args = parse_args()
+    seed_everything(args.seed)
+    vg_dir = Path(args.vg_dir)
+    truth = [json.loads(line) for line in
+             (vg_dir / "ground_truth.private.jsonl").read_text().splitlines() if line.strip()]
+    vision = {json.loads(line)["node_id"]: json.loads(line) for line in
+              (vg_dir / "vision_nodes.private.jsonl").read_text().splitlines() if line.strip()}
+    records = {json.loads(line)["node_id"]: json.loads(line) for line in
+               (vg_dir / "text_nodes.jsonl").read_text().splitlines() if line.strip()}
+
+    pairs = [p for p in truth
+             if p["vision_node_id"] in vision and p["text_node_id"] in records]
+    needed = pairs[: args.samples + args.fit_scenes]
+    print(f"{len(needed)} scenes", flush=True)
+
+    processor = AutoImageProcessor.from_pretrained(args.model)
+    model = AutoModel.from_pretrained(args.model, torch_dtype=torch.float32).to(args.device)
+    model.eval()
+    mean = torch.tensor(processor.image_mean).view(3, 1, 1)
+    std = torch.tensor(processor.image_std).view(3, 1, 1)
+
+    # one crop per annotated region, in the order the phrases are given
+    crops, owners = [], []
+    for index, pair in enumerate(tqdm(needed, desc="crop")):
+        record = vision[pair["vision_node_id"]]
+        phrases = records[pair["text_node_id"]]["region_closed"]
+        path = Path(args.image_cache, f"{record['source_image_id']}.jpg")
+        if not path.exists():
+            continue
+        with Image.open(path) as handle:
+            picture = handle.convert("RGB")
+        width, height = picture.size
+        for position, region in enumerate(record["regions"][: len(phrases)]):
+            x0 = max(0, min(int(region["x"]), width - 2))
+            y0 = max(0, min(int(region["y"]), height - 2))
+            x1 = max(x0 + 2, min(int(region["x"] + region["width"]), width))
+            y1 = max(y0 + 2, min(int(region["y"] + region["height"]), height))
+            patch = picture.crop((x0, y0, x1, y1)).resize(
+                (args.crop_size, args.crop_size), Image.BILINEAR
+            )
+            crops.append(torch.from_numpy(np.asarray(patch).copy())
+                         .permute(2, 0, 1).float() / 255.0)
+            owners.append((index, position))
+
+    features = []
+    for start in tqdm(range(0, len(crops), args.batch_size), desc="encode"):
+        batch = torch.stack(crops[start : start + args.batch_size])
+        normalised = ((batch - mean) / std).to(args.device)
+        hidden = model(pixel_values=normalised, return_dict=True).last_hidden_state
+        features.append(hidden[:, 0].float().cpu().numpy())
+    features = np.concatenate(features)
+    print(f"{len(features)} region crops encoded", flush=True)
+
+    vectors = word_vectors(load_phrases(vg_dir / "text_nodes.jsonl", "region_closed"), args)
+
+    def phrase_vector(phrase: str) -> np.ndarray | None:
+        hits = [vectors[t] for t in re.findall(r"[a-z]+", phrase.lower()) if t in vectors]
+        return np.mean(hits, axis=0) if hits else None
+
+    # collect per scene, keeping only regions whose phrase encodes -- so the
+    # two sides carry exactly the same parts in exactly the same order
+    vision_parts: dict[int, list] = {}
+    text_parts: dict[int, list] = {}
+    for (index, position), feature in zip(owners, features):
+        phrase = records[needed[index]["text_node_id"]]["region_closed"][position]
+        encoded = phrase_vector(phrase)
+        if encoded is None:
+            continue
+        vision_parts.setdefault(index, []).append(feature.astype(np.float64))
+        text_parts.setdefault(index, []).append(encoded)
+
+    usable = [i for i in sorted(vision_parts) if len(vision_parts[i]) >= 4]
+    evaluated = [i for i in usable if i < args.samples][: args.samples]
+    fit = [i for i in usable if i >= args.samples]
+    print(f"{len(evaluated)} evaluated, {len(fit)} for fitting", flush=True)
+
+    results = {}
+    for name, parts in (("vision", vision_parts), ("text", text_parts)):
+        fitted = content_directions([np.stack(parts[i]) for i in fit], args.shrinkage)
+        centre, basis, values = fitted
+        width = min(args.keep, basis.shape[1])
+        scale = values[:width] ** args.weight_power
+        results[name] = [
+            F.normalize(torch.tensor(
+                (np.stack(parts[i]) - centre) @ basis[:, :width] * scale,
+                dtype=torch.float32), dim=-1)
+            for i in evaluated
+        ]
+
+    visual_field = standardise(moment_field(results["vision"]).double().numpy())
+    text_field = standardise(moment_field(results["text"]).double().numpy())
+    mask = ~np.eye(len(evaluated), dtype=bool)
+    summary = {
+        "protocol": (
+            "DIAGNOSTIC, NOT A METHOD. Each annotated region box is encoded as a "
+            "crop and paired with its own description, so the two modalities "
+            "carry the same parts in the same order. That box-to-phrase link is "
+            "cross-modal supervision a deployed system would not have."
+        ),
+        "scenes": len(evaluated),
+        "mean_parts": float(np.mean([len(vision_parts[i]) for i in evaluated])),
+        "correlation": float(np.corrcoef(visual_field[mask], text_field[mask])[0, 1]),
+        "anchor_bound_with_part_oracle": anchor_bound(visual_field, text_field),
+        "anchor_bound_unsupervised_reference": 0.336,
+        "within_modality_ceiling_text": 0.805,
+        "reading": (
+            "A large jump means part correspondence is the whole problem and is "
+            "an engineering target. Staying near 0.336 means the two encoders "
+            "do not covary at the part level either, and no aggregation "
+            "rescues this corpus."
+        ),
+    }
+    print(json.dumps(summary, indent=2))
+    write_json(args.output, summary)
+
+
+if __name__ == "__main__":
+    main()