diff options
Diffstat (limited to 'RANK_RESULTS.md')
| -rw-r--r-- | RANK_RESULTS.md | 67 |
1 files changed, 65 insertions, 2 deletions
diff --git a/RANK_RESULTS.md b/RANK_RESULTS.md index d520e5e..ffee8c9 100644 --- a/RANK_RESULTS.md +++ b/RANK_RESULTS.md @@ -1,7 +1,16 @@ # The gate was measuring the wrong thing -*2026-08-01. Supersedes the correlation threshold used as the project's -go/no-go statistic since the synthetic world results.* +*2026-08-01. Retires the correlation threshold that has been the project's +go/no-go statistic since the synthetic world results, records two replacement +statistics that were proposed and refuted the same day, and settles on a third +that survives its controls: an upper bound obtained by giving the solver half +the correspondence and asking it to recover the rest.* + +**Result in one line.** The correlation does not govern recovery in either +direction; what does is whether a representation can identify scenes at all, +measured by matching against anchors whose correspondence is given. Visual +Genome scores 0.291 on that bound against 0.98 for the synthetic world that +recovers, and its blind recovery is 0.0000 against 0.0039 chance. ## The correlation does not govern recovery @@ -102,6 +111,60 @@ than everything found (recoverable), tied with what is found (symmetry), or shallower than what is found (information deficit). The gate is running on both fields; the sections below should be read against this correction. +## Resolution: ask the matching question, and give one thing away + +Three statistics failed the same way — fields agreeing on the number and +disagreeing on recovery — and the pattern in the failures is that each was +invented by staring at the fields rather than by asking what matching needs. +The one that works asks directly, with a single concession. + +Declare half the scenes anchors and hand over their correspondence. Describe +every remaining scene, in each modality, by its field row against the anchors. +Match the two descriptions one-to-one by Hungarian assignment. Nothing is +searched; the answer is a linear program, and it costs seconds. + +This **upper-bounds blind recovery**, because blind recovery must additionally +discover the anchor correspondence. It held in every field measured: + +| Field | ρ | anchor bound | blind recovery | search gap | +|---|---|---|---|---| +| synthetic, full | 0.928 | 0.989 | 0.958 | +0.031 | +| synthetic, noise 0.35 | 0.828 | 0.984 | 0.947 | +0.037 | +| synthetic, caption omits size | 0.830 | **0.997** | 0.056 | **+0.941** | +| synthetic, rank 8 | 0.906 | 0.930 | 0.129 | +0.801 | +| Visual Genome, best | 0.731 | **0.291** | 0.000 | +0.291 | +| Visual Genome, baseline | 0.656 | 0.195 | 0.000 | +0.195 | + +**The bound splits the deficit in two, and the two halves call for opposite +work.** The bound itself says whether the representation identifies scenes at +all; the gap between bound and blind recovery says whether the search can find +what the representation contains. + +Read down the table and the three regimes are unmistakable. The two recovering +fields have a bound near one and a gap of three points — representation good, +search working. Caption-omission and rank-8 truncation have a bound of 0.93 to +**0.997** and lose almost everything to the search: the information is intact, +perfectly so in the omission case, and blind matching cannot reach it. Visual +Genome is the opposite failure — its bound is 0.291, so **even with the +correspondence handed over, seven scenes in ten cannot be identified**, and no +solver improvement addresses that. + +This corrects two things I wrote earlier today. The caption-suppression ladder +does not show information being destroyed; it shows blind searchability being +destroyed while the information stays. Suppressing a factor makes scenes that +differed only in that factor interchangeable *under permutation*, which the +given anchors break and blind matching does not. And Visual Genome's problem +was never the width of its shared spectrum — it is that a region description +and a patch descriptor, compared through relation profiles, identify a +photograph about 29% of the time. + +The bound is an empirical upper bound rather than a proven one: handing over +anchors removes search but also changes the estimation problem. It was not +violated in six fields spanning the full range of outcomes, and unlike the +three refuted statistics it separates every case they collapsed. It is now the +project's cheap gate, and a representation that cannot pass it should not be +searched. + ## What the shared spectrum measures Each field is eigendecomposed and the principal angles between the two leading |
