summaryrefslogtreecommitdiff
path: root/RECOVERY_RESULTS.md
blob: 00125df1375d4340fe0381ea2fa70ea097ae7cd6 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
# Blind recovery on gate-passing states: the search phase

Run date: 2026-07-29

## Question

The basin audit licenses recovery: on content-projected VG states the true
assignment sits 5--19% below every quench, so a search that reaches the
bottom of the landscape would recover it. This experiment asks whether
three search families get there without any pair information. The text
side enters through a hidden shuffle; hidden truth scores outcomes only.

## Arms

- Parallel tempering: eight-replica Metropolis over permutations with
  exact closed-form swap deltas and neighbor exchange, 60,000 rounds.
- Entropic Sinkhorn annealing: soft coupling, barycentric states,
  temperature and entropy schedules, four restarts.
- Spectral-band homotopy: the Sinkhorn arm warm-started through
  progressively finer content bands (4 to full dimensions), a spectral
  proxy for band-limited continuation on the symmetric group.

## Result: the energy is right and the search is the wall

On all three gate-passing conditions (512 nodes, subset seed 0):

| Condition | Gate margin | Tempering: E over true / accuracy | Sinkhorn | Homotopy |
|---|---:|---:|---:|---:|
| VG isolated, k=256 | +7.8--12.0% | +9.4% / 3.1% | +72% / 0.4% | +62% / 0.4% |
| VG isolated, k=128 | +5.1--10.8% | +10.1% / 2.1% | +100% / 0.2% | +76% / 0.6% |
| VG in-context, k=128 | +11.6--18.5% | +19.8% / 1.0% | +99% / 0.8% | +78% / 0.0% |

Tempering equilibrates into exactly the quench band of the gate report --
the glass's low shelf -- and never enters the true funnel; chance is 0.2%,
so 1--3% accuracy is a population-level shift, not recovery. The soft
arms stall far higher; their couplings put no verified mass anywhere
(top-margin-decile precision 0.000 in all runs, mutual-NN precision below
1.2%). Continuous relaxation loses to discrete annealing everywhere, and
discrete annealing loses to the landscape.

The picture matches the hard phase of planted-assignment problems: the
configuration is information-theoretically identifiable (the gate), while
local dynamics equilibrate into an exponential shelf of wrong states
whose energies sit just above the planted one. The distance between the
statistical and the algorithmic threshold is now the project's central
quantity.

## What moves the algorithmic threshold

Ordered by expected yield, per the tool inventory of
`STRUCTURE_DESIGN.md`:

1. Constraint density per node: the N-scaling probe below, and at fixed N
   the view-level recursion and R8 intervention constraints, which
   sharpen the funnel rather than the search.
2. Side information: gauge-free unary anchors (counting, mm-space
   observable invariants) that bias the sampler into the funnel; in
   planted problems even weak unary priors change the search class.
3. Message passing: cavity/BP dynamics for planted matching reach partial
   recovery at densities where annealing fails; this is the queued
   spin-glass tool.

## N-scaling probe

Tempering-only runs at N=1024 and N=2048 with a fixed 80,000-round budget
test whether the phase boundary approaches with constraint density. Best
replica by energy:

| N | E over true | Accuracy | Chance |
|---:|---:|---:|---:|
| 512 | +9.4% | 3.1% | 0.20% |
| 1,024 | +5.2% | 2.5% | 0.10% |
| 2,048 | +15.0% | 0.05% | 0.05% |

Doubling to 1,024 cuts the energy gap by nearly half at constant budget.
The under-equilibrated 2,048 point is budget-confounded, so an
equal-moves-per-node rerun with 320,000 rounds was run:

| N | Moves per node | E over true | Accuracy |
|---:|---:|---:|---:|
| 512 | ~117 | +9.4% | 3.1% |
| 1,024 | ~78 | +5.2% | 2.5% |
| 2,048 | ~156 | +10.4% | 0.05% |

At matched per-node budget the reachable shelf returns to the same
+9--10% band as N=512, and the small accuracy signal present at 512 and
1,024 vanishes entirely. One seed per point, so the 1,024 dip may be
noise; what the curve rules out is the optimistic reading -- **no
algorithmic threshold is approached by density alone within the tested
range** (N up to 2,048, 320k rounds). The recovery bottleneck stands, and
the priority order shifts to the other two levers: gauge-free unary side
information and cavity-style message passing, with constraint sharpening
(view-level recursion, interventions) as the structural route.

## Unary anchors, first wave: coarse anchors do not move the search

The C5 battery extracted gauge-free unary anchors with no model passes:
numeral content, color-lexicon histograms, chromatic fraction, size and
light words on the text side; hue-band histograms, chromatic fraction,
luminance, and box areas on the vision side, with the lexicon-to-physics
maps frozen before evaluation. The combined anchor field has real
population-level signal at the true pairing (z = 17.2) and no per-pair
reliability: median truth rank 1,969 of 5,000, top-1%-margin precision
2%, mutual-NN precision 0.17%.

Injected into the tempering energy as a dense unary field, it strictly
degrades recovery, monotonically in the weight (best-by-relational-energy
replica):

| Unary weight | Init | Accuracy | Relational E over true |
|---:|---|---:|---:|
| 0 (baseline) | random | 3.1% | +9.4% |
| 0.5 | random | 1.6% | +30.2% |
| 2.0 | random | 2.0% | +76.2% |
| 2.0 | anchor-Hungarian | 1.0% | +82.6% |
| 8.0 | random | 1.0% | +115.2% |

The mechanism echoes the spectral audit: scene-level color, luminance,
and size are coarse attributes that already live inside the shared
topical subspace, so the field adds a noisy copy of information the
relational energy has, plus its own wrong optima for the sampler to
compromise with. Side information can move a hard phase only if it is
per-pair reliable (sparse verified seeds -- there are none to sparsify
here) or statistically independent of the relational field. Pixel and
lexicon statistics at node level are neither.

The constructive redirect: fine-grained anchors live at view level, where
region phrases carry numerals and spatial predicates about individual
regions, boxes carry geometry, and the candidate set is sixteen rather
than five hundred. The second anchor wave belongs inside the view-level
recursion, not the node-level search; message passing with priors remains
the queued search-side tool.

## Unary anchors, second wave: view-level anchors yield the first seeds

Per-view anchors -- twelve-class color lexicon against crop pixel classes
(white, black, gray, and brown rules on value and saturation carry most
of the coverage), size words against within-scene box-area ranks, light
words against crop luminance, position words against box centers -- give
the first per-pair-reliable signal of the project:

- frame-free view matching 11.5% against 6.25% chance; 16.9% on the
  35,484 views whose phrase carries an attribute (2.7x);
- **seed calibration: 35.6% precision on the top-1%-margin views** (354
  seeds at 5.7x chance), against 0--4% for every prior high-confidence
  channel;
- position words are nearly absent from region phrases (coverage under
  1.1%), so color and light do the work.

Aggregated to node level (Hungarian value of the per-pair 16-view anchor
matrix as node affinity, three subset seeds at 512): retrieval R@10
8.6--9.6% against 1.95% chance (4.4--4.9x, the best frame-free node
signal to date; graph signatures reached 3.1--3.5x), z = 16--17, median
rank 113--123 of 512. Node-level seed precision stays at 2--6%: one
order below seeding threshold. Adding fixed-assignment relational
agreement trades recall for precision without changing the order.

Pixel and lexicon statistics are now exhausted at both levels. The two
open moves are the numeral-to-repetition anchor (patch-level extraction,
untested) and the aggregation everything above points to: a factor graph
over node and view assignments with anchor unaries and relational
pairwise terms, solved by message passing, so that 35%-precision view
seeds, 5x node affinities, and the gate-passing relational energy
reinforce globally instead of being consumed by independent per-node
Hungarians.

## Main artifacts

- `artifacts/manifold_gate/recovery_vg_k{128,256}_seed0.json`
- `artifacts/manifold_gate/recovery_vg_ctx_k128_seed0.json`
- `artifacts/manifold_gate/recovery_scale_n{1024,2048}.json`
- `artifacts/manifold_gate/recovery_scale_n2048_long.json`
- `artifacts/manifold_gate/anchor_battery.json`
- `artifacts/manifold_gate/anchor_unary.pt`
- `artifacts/manifold_gate/recovery_unary_lam{0.5,2.0,8.0}.json`
- `artifacts/manifold_gate/recovery_unary_lam2_unaryinit.json`
- `artifacts/manifold_gate/view_anchor_battery_v{2,3}.json`
- `artifacts/manifold_gate/view_anchor_features.pt`
- `artifacts/manifold_gate/view_anchor_affinity{,_v2}.json`