diff options
Diffstat (limited to 'NATURAL_RESULTS.md')
| -rw-r--r-- | NATURAL_RESULTS.md | 50 |
1 files changed, 50 insertions, 0 deletions
diff --git a/NATURAL_RESULTS.md b/NATURAL_RESULTS.md index e0ea4b7..502acb3 100644 --- a/NATURAL_RESULTS.md +++ b/NATURAL_RESULTS.md @@ -135,3 +135,53 @@ observation is necessary now has a natural-data counterpart to test. - `artifacts/vg_5k/natural_families{,_dist}.json` - `artifacts/vg_5k/natural_objects{,_16,_oracle}.pt` - `artifacts/vg_5k/natural_fields{,_16}.pt` + +--- + +# The corpus is capped, and the part correspondence was never the problem + +*Appended 2026-08-01.* + +Six representational interventions were tried on Visual Genome across one +session — a wider weighted projection, larger text vectors, structured phrase +encoding, more segments, annotated boxes instead of segmentation, third-order +moments in the set kernel, and a backbone at twice the depth. They moved the +field correlation from 0.656 to 0.731 and the cross-modal anchor bound from +0.195 to 0.336. None was chosen against a ceiling, because no ceiling had been +measured. That was the same error being made on the matching side at the same +time: **optimising without a bound.** + +Three measurements now bound it, and they disagree with where the effort went. + +| Measurement | Value | +|---|---| +| Vision identifies its own scenes (split-half) | **0.994** | +| Text identifies its own scenes (split-half, IDF bag-of-words) | **0.805** | +| Text identifies its own scenes (our encoding pipeline) | 0.491 | +| Cross-modal, unsupervised parts | 0.336 | +| **Cross-modal, annotator-supplied part correspondence** | **0.359** | + +Each region box in Visual Genome is index-aligned with its own description, so +the crop inside the box can be encoded and paired with the phrase that +describes it — putting the two modalities' parts in correspondence by +annotation, which is cross-modal supervision a deployed system would never +have. **It buys 0.023.** + +That settles the question the pipeline work could not. The two encoders do not +covary at the part level either: a DINOv2 descriptor of a region and an English +phrase describing the same region do not agree about what makes two regions +alike, and no segmentation, aggregation, kernel or backbone repairs that. The +0.805 within-text ceiling was never the cross-modal ceiling; the cross-modal +ceiling for this encoder pair on this corpus is about **0.36**, against the 0.9 +that recovery needs. + +**Visual Genome is closed.** Not because the method fails but because this +pairing of encoder and corpus does not carry the agreement the method consumes, +and that is now a measurement rather than a suspicion. + +What it leaves is a screening protocol rather than a repair list. Three numbers, +all cheap and all computable before any pipeline is built: each modality's +split-half self-identification, the cross-modal anchor bound, and — where the +corpus offers aligned parts — the same bound with part correspondence given. +A corpus whose cross-modal bound sits near its within-modality ceilings is worth +building on; Visual Genome's sits at 45% of the lower of them. |