1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
|
# WorldAlign research log
This file records research decisions, failed hypotheses, and the current
experimental gate. Exact metrics live in the result reports and machine-readable
artifacts. The collaborator-facing statement of the project is maintained in
`PROJECT_CONCEPT.md`.
## Research target
The target is a real multimodal large-model system, not a geometry result or a
toy correspondence benchmark.
The working hypothesis is that independently trained, sufficiently informative
unimodal representations may recover the same world structure up to a change
of coordinates. Multimodal alignment could then be obtained by descending a
high-level shared-world energy, without paired contrastive supervision.
Important constraints:
- Do not treat the word "graph" as a commitment to explicit semantic nodes.
It is only an analogy for richly constrained relations in learned states.
- Node-wise or sample-wise sufficiency is not enough. The observations must be
overcomplete enough to constrain relations among world states.
- Generic encyclopedia text is not a closed counterpart of the visible world:
it contains many latent factors that images cannot identify.
- The target method is not an image-to-language MLP bridge. The bridge baseline
is useful only as an interface/control.
- Paired examples may be used for held-out diagnosis, never by the unpaired
energy, optimizer, initialization, model selection, or stopping rule.
## 2026-07-28: learned-bridge baseline
Flickr30k, frozen DINOv2 and frozen Qwen were aligned with static
GW/SWD/isometry objectives.
- Cross-modal relational similarity is measurably non-random.
- Static GW produces sharp but mostly incorrect correspondences.
- A paired bridge verifies that the frozen-language-model interface works.
- The unpaired learned bridge does not produce usable multimodal ability.
See `RESULTS.md`.
## 2026-07-29: overcomplete observations
Five independent captions of the same scene were treated as an observation
orbit. On 512 held-out examples, the correct-vs-shuffled relation-energy gap
increased from 0.316 with one caption to 0.582 with five captions.
This supports the narrower claim that relational overcompleteness makes the
correct configuration easier to distinguish. It does not establish that the
current energy identifies the correct absolute configuration.
## 2026-07-29: energy descent without a cross-modal map
Free Qwen-side latent particles were initialized from real text states and
optimized using:
- standardized relation-field matching;
- multiscale conditional-neighborhood matching;
- sliced text-population matching;
- proximity to text-only prototypes.
Weak manifold regularization briefly improved held-out R@10 from 2.34% to
12.11%, but left the decodable language manifold. Strong regularization
improved it only to 6.25% at the best intermediate step.
Language-prior-only and shuffled-vision controls showed that caption-generation
improvements were not reliably image-conditioned.
## 2026-07-29: convergence audit and falsification
The original logged total was not a fixed objective because relation and
conditional weights were annealed. Therefore that curve could not establish
convergence.
Starting from the strong-manifold (`m30`) endpoint, the final weights were
frozen and optimization was continued at one tenth of the learning rate:
- fixed energy decreased from 2.767 to approximately 1.54;
- R@10 decreased from 5.86% to approximately 3--4%;
- median rank worsened from 103 to approximately 120--123;
- random R@10 for 256 candidates is 3.91%;
- gradients remained nonzero and Adam continued to oscillate.
Most decisively, the true paired language configuration has fixed energy about
4.49, while an incorrect synthetic configuration reaches about 1.54.
Therefore the present energy is falsified as an alignment objective:
\[
E(\text{correct paired state}) >
E(\text{incorrect synthetic state}).
\]
The early retrieval gain was a trajectory transient, not evidence that further
energy descent would recover the correct gauge. More steps or optimizer tuning
cannot repair this ordering.
## 2026-07-29: on-manifold assignment gate
Decision, confirmed with the project owner: restrict the configuration space
to permutations of real frozen text states. On this space the language-only
terms of the falsified energy are permutation-invariant constants, and the
continuous counterfeit route is closed by construction. Per the standing
gate, only ranking and local-ordering diagnostics were run; no new descent.
Result: the falsification survives the restriction.
- Global ordering always passes: the true assignment ranks below all 1,000
random permutations in every condition (z 2.7--18.7).
- Local ordering always fails: 6.8--33% of transpositions of the true
assignment lower the energy (exact closed-form enumeration); at N=5,000
the count is 2.70M of 12.50M. Scale does not repair it.
- Exact steepest 2-swap descent from the truth keeps 2.7--13.9% of nodes;
random-start quenches reach energies 0.9--1.9 below the true
configuration at 0.0--0.4% accuracy. On-manifold counterfeits exist in
every condition.
Mechanism, from three audits: pooled states are extremely anisotropic
(99.5% of Flickr caption pairs above cosine 0.8); the cross-modal share of
standardized relational structure is Spearman 0.19--0.31; that share is
spectrally concentrated (top-16 population directions: 0.367; full space:
0.306; whitening collapses it to 0.03). Five-caption orbits and visible
relations denoise within this coarse subspace; distribution-valued edge
channels (std/q10/q90 of view-pair cosines) carry no shared signal and
dilute the energy; model scale was already known to be inert. A roughly
sixteen-dimensional topical code cannot separate 512 instances, so no
energy over these node states can identify the assignment.
See `MANIFOLD_RESULTS.md`.
## 2026-07-29: view-level shared structure inside VG nodes
The per-node view permutation hidden at preparation time is exactly
replayable from the seed and the private image IDs; the replay reconstructs
all 5,000 released phrase bundles verbatim and is used only for evaluation.
- Within-node relation fields align across modalities: matched Spearman
0.128 against -0.002 shuffled, z = 46.6 over 5,000 nodes.
- With the true node-level alignment as a coarse frame, population-context
profiles match the 16 views at 14.5% accuracy against 6.25% chance
(top-3 29.5%); a larger frame helps (200 nodes: 12.9%). This is an upper
bound: a blind system must substitute its own recovered coarse frame.
- Within-node relations alone stay near chance (7.0%), so the population
context carries the signal.
The fine-grained shared structure missing at node level exists one level
down, in the compositional structure that pooling discards.
## 2026-07-29: Ricci-flow control and static view-level gate
Two same-day continuations, both requested before the structural redesign.
Ollivier-Ricci flow on each modality's kNN geometry (independent,
identical hyperparameters, diffusion-only baseline for attribution) more
than doubles the global ranking z (Flickr 21.4 to 47.8; VG 6.9 to 22.5)
and changes no verdict: 9--14% improving transpositions, descent retention
1--5%, counterfeits at about half the true energy in all four conditions.
Geometric smoothing is a frame ingredient, not a resolver; it cannot be
part of the alignment transform itself (information loss), matching the
whitening result.
The static view-level gate fails at scale: with the true coarse frame,
none of 5,000 nodes has the true view assignment as a strict local
minimum (32.0% improving for the within-node relational energy, 28.3% for
the profile energy, 25.5% joint). Frame-free internal-field matching is
null at 512 nodes over three seeds (R@10 at chance, zero margin-decile
precision, true-pair z 0.1--1.8 after a capacity null; the negative z
seen at 96-node smoke scale did not replicate). Consequence: the view
level needs model-internal relations or true re-encoding interventions --
the R3/R8 battery in `STRUCTURE_DESIGN.md` is the critical path.
## 2026-07-29: representation batteries and the first passed basin audit
The R6 and R3 batteries of `STRUCTURE_DESIGN.md` were run the same day.
R6, within-scene noise whitening: directions maximizing between-scene
over within-scene variance, fitted per modality on unpaired orbits
(train-only for Flickr, holdout-only for VG). Cross-modal relational
Spearman rises 0.298 to 0.358 (Flickr orbit), 0.189 to 0.326 (single
caption, now above the raw orbit mean), 0.213 to 0.614 (VG both sides,
k=8); improving-swap fractions collapse 7.8% to 0.5% and 23.5% to 0.25%.
Population whitening had destroyed the same signal; the only difference
is whitening the within-scene rather than the total covariance.
R3, model-layer readout: joint-context encoding on all 5,000 VG nodes.
Attention weights are modality-specialized (best attention-attention
matched rho 0.076); in-context states carry rho 0.216 against the 0.128
isolated baseline (z 69.6 vs 46.6). Relations do live in the model layer,
and the shared readout is the mixed states, not the mixing weights.
Full gate on the new states: nine of nine VG conditions pass the basin
audit -- ten random-restart quenches per run all end 5--19% above the
true energy, replicated over subset seeds 0--2, for both isolated
projected states and the in-context composite (which roughly doubles the
margin at matched k). First energies in the project whose descent finds
nothing below the truth. Flickr still fails by under 1%: one visual view
means no visual orbit to whiten, so two-sided multi-view observation is a
data-protocol requirement. The true assignment is still not a strict
local minimum (0.17--0.28% improving swaps; descent keeps 52--59%), so
what is licensed is soft-coupling recovery, not hard matching.
See `BATTERY_RESULTS.md`.
## 2026-07-29: blind recovery enters the hard phase
First Phase-2 experiment on the gate-passing states, hidden-shuffle
protocol, three search arms. Parallel tempering (eight replicas, exact
closed-form deltas, 60k rounds) equilibrates into the quench band of the
gate report -- ending 9--20% above the true energy at 1--3% accuracy --
and never enters the true funnel; entropic Sinkhorn and spectral-band
homotopy stall at +62--100% with zero verified seed mass. Discrete
annealing beats continuous relaxation everywhere and still loses to the
landscape.
Diagnosis: the classic hard phase of planted assignment -- the gate shows
the configuration is information-theoretically identifiable while local
dynamics equilibrate into an exponential shelf of wrong states. The gap
between the statistical and algorithmic thresholds is now the central
quantity.
The N-scaling probe closes the cheapest hypothesis: at matched per-node
move budget (N=2,048, 320k rounds), the reachable shelf returns to the
same +9--10% band as N=512 and the small accuracy signal vanishes.
Density alone does not approach an algorithmic threshold in the tested
range.
The first C5 anchor wave closes the second-cheapest. Pixel- and
lexicon-derived unary anchors carry population signal (combined z = 17.2)
with no per-pair reliability (top-margin precision 2%, mutual-NN 0.17%),
and dense injection into the tempering energy degrades recovery
monotonically in the weight (accuracy 3.1% to 1.0--2.0%, relational
energy +9% to +30--115%). Scene-level anchors are coarse-redundant with
the shared subspace; usable side information must be per-pair reliable or
independent of the relational field. Fine-grained anchors (numerals,
spatial predicates, box geometry) live at view level with sixteen
candidates, so the second wave merges into the view recursion. Remaining
search-side lever: cavity/message-passing dynamics.
The second wave (2026-07-30) delivers the first seeds. Per-view color,
light, size, and position anchors against crop pixels and box geometry:
frame-free view matching 11.5% vs 6.25% chance, 35.6% precision on
top-1%-margin views (354 seeds; every prior high-confidence channel was
0--4%). Aggregated per-pair Hungarian values give the best frame-free
node affinity to date (R@10 4.4--4.9x chance over three seeds), with
node-level seed precision still at 2--6%. Pixel and lexicon statistics
are exhausted; the next package is a factor graph over node and view
assignments -- anchor unaries, relational pairwise, message passing --
plus the untested numeral-to-repetition anchor.
See `RECOVERY_RESULTS.md`.
## 2026-07-30: synthetic closed world v0 -- the objective becomes the bottleneck
A procedural world with exact closure (43-word vocabulary, unique
referents, ring-placement multiplicity, relation-constrained rendering,
ground-truth interventions) and from-scratch unimodal towers on disjoint
splits. First measurement inverts the natural-data pattern: linear CKA
matches (0.278 vs 0.262) while relational Spearman collapses six-fold
(0.030 vs 0.184, z 2.4 vs 12.3). The gate fails accordingly; content
projection restores ranking (z = 117) but not the basin (counterfeit at
-13%).
Attribution is clean because data sufficiency is total by construction:
InfoNCE's uniformity pressure turned the vision tower into a
near-orthogonal instance codebook, erasing between-scene geometry.
Sufficiency of the data does not transfer to sufficiency of the
representation; the unimodal objective decides which world geometry
survives, and R1 is a property of the data-objective pair. The SSL
objective is now a phase-diagram axis. Next: reconstruction-family
towers (doubling as the masked predictors of the prediction-transfer
energy), then the structure identification battery.
See `SYNTH_RESULTS.md` and the 2026-07-30 additions to
`STRUCTURE_DESIGN.md` (unified element-relation object; the three-rung
energy ladder ending in lifted unimodal world models).
## 2026-07-30/31: objective battery, set machinery, and the first phase coordinates
The synthetic world's second day closes three questions and opens one.
Objective battery, complete: five standard self-supervised recipes
(orbit-InfoNCE, SimMIM, their hybrid, data2vec-lite, slot attention at
two resolutions) all fail to produce factor-complete global states, each
by a distinct mechanism -- shortcut selection, local-only knowledge,
non-merging losses, softened shortcut, and spatial tiling instead of
object binding. Slot attention carries the most relational sharing
(0.128 against 0.030 for InfoNCE) and the set-versus-pool doctrine is
confirmed quantitatively: pooling slot sets without correspondence drops
color decodability from 0.99 to 0.55.
Set-kernel machinery, taken to a hand-crafted ceiling
(connected-component descriptors against phrase bag-of-words): seven
measured iterations moved the gate from z = -0.1 and 45% improving swaps
to z = 51, 0.15% improving swaps, and 77% descent retention. Priced en
route: mass-weighting inside matching destroys grading; multi-view field
averaging cancels segmentation noise; a factor present on one side only
(shape) is a counterfeit lever; so is size-dependent matching bias.
End to end, honestly: blind tempering on the best fields still finds
assignments 4--9% below the truth at chance accuracy, and size
residualization removes shared signal along with the nuisance. With VG's
passing configuration at field validity 0.61 and this ceiling near 0.45,
the basin-pass boundary at N = 512 is bracketed between them: the first
coordinates of the identifiability phase diagram.
See `SYNTH_RESULTS.md`.
## 2026-07-31: the Fock lift and the (validity, N) boundary
The out-of-the-box program's first invention went to flight the same
morning. A symmetric-tensor moment kernel on object sets (the truncated
second-quantized lift: composition as algebra, multiplicity as an
observable, no matching step and hence none of the measured matching
biases) with factor-mirrored one-hot descriptors gives the synthetic
world its first full basin pass at N = 64 -- no counterfeit, 92%
retention. At N = 512 counterfeits return 18--22% below the truth and
recovery stays at chance; residual segmentation error (30% wrong counts)
and shape-cluster impurity suffice at eight times the configuration
space. Identifiability is now measured as a joint boundary in field
validity and population size: pass at (0.45, 64), fail at (0.45, 512),
pass at (0.61, 512). Queued inventions, in order: orbit-derived
per-entry precision weighting (channel-matched energy), third-order
triangle invariants (holonomy against pairwise counterfeits), moment
ideals as gauge-free content coordinates; mixed-curvature products noted
as a symmetry-breaking device and deprioritized.
See `SYNTH_RESULTS.md` and the tool inventory in `STRUCTURE_DESIGN.md`.
## 2026-07-31: the gate statistic was measuring the searcher
A methodological correction that supersedes part of the same day's
optimism. Third-order (triangle) invariants were added and, in a harness
whose descent proposes 64 sampled pairs per step, the truth became
essentially immovable at N=512 (retention 99.6%, no counterfeit, z=50),
replicated over three hidden shuffles. Two controls dismantled the
reading: the pairwise-only energy passes identically in that harness
(retention 100%), and long tempering on the triangle energy at N=128
reaches 10% below the truth at zero accuracy.
Conclusion: descent retention measures the strength of the search
operator, not the identifiability of the energy. Exact all-pairs
enumeration (yesterday's harness) is a strictly stronger searcher than
sampled proposals, and long tempering is stronger still. The gate
statistic is therefore redefined:
\[
E(\text{truth}) \le E(\text{deepest state a strong searcher reaches}),
\]
measured with long parallel tempering from random starts plus a
truth-initialized arm, uniformly across candidate energies. Retention and
improving-swap fractions remain as diagnostics, never as verdicts. All
candidate energies are being re-measured on this basis
(`synth_deep_gate.py`); readings from the weaker harness are marked
superseded in `SYNTH_RESULTS.md`.
## 2026-07-31: closed form, spectral solvers, and a clean Tier 0
Three results and one protocol decision, all same day.
Closed form: the triangle energy equals a constant minus 2 trace(M^3)/6
with M the elementwise product of permuted text and visual fields, so
one batched matrix product scores hundreds of permutations over all
C(N,3) triples. Validated to 3e-7 against brute force; opens trace(M^k)
for any k. Iteration time per configuration falls from hours to minutes.
Spectral solvers (Umeyama, GRAMPA) were added because they are
polynomial and dimension-free -- relation fields are N x N whatever the
embedding widths -- and therefore bypass the glassy landscape. They
return 3--13x chance. Correlated-graph-matching theory explains it: at
N=256 the information-theoretic threshold sits near rho = 0.29 and the
polynomial threshold near rho = 0.9, and our fields are at 0.51--0.57.
The project is squarely in the computationally hard phase, which now has
a quantitative account rather than an empirical one.
Identifiability is not correlation. The colour-only field reaches rho =
0.863 yet caps at 49% because only 126 of 256 scenes have distinct
colour multisets; its eigenvalue degeneracy is that collision structure.
Full descriptors are 100% distinct. Channel mixing inside one moment
kernel dilutes 0.86 to 0.51.
Protocol, decided with the user: anchors that declare "red means this
hue" are not labels but are hand-supplied cross-modal prior, so results
must be reported by tier. Tier 0 forbids any declared correspondence.
Achieved the same day on world v1 (Zipf-skewed factor marginals, since
uniform v0 makes the correspondence information-theoretically
unrecoverable): text factor families discovered by mutual exclusivity,
vision colour classes by k-means on raw RGB, the two paired by marginal
frequency rank -- 10/10 entries semantically correct, fields at rho =
0.570 with a 97.3% identifiability ceiling.
See `SYNTH_RESULTS.md`.
## 2026-07-31: first recovery, and it was segmentation all along
The rho decomposition settled the target: oracle segmentation gives field
correlation exactly 1.0 and the Tier 0 derived dictionary is free (oracle
and derived fields are identical), while connected-component segmentation
gives 0.5696. Three unimodal fixes closed most of it -- Gestalt
appearance grouping (exact group count 71.9 to 90.2%),
distance-transform watershed for touching ring members (exact object
count 74.2 to 100%), and one-dimensional k-means on radial extent for
size classes (52.7 to 87.9%, because area confounds size with shape and
the classes are gap-separated rather than equally populated).
Field correlation went 0.570 to 0.671 to 0.928, and GRAMPA recovery
followed it 5.1% to 9.4% to **53.9%** at N=256 (0.39% chance), replicated
at 53.9/53.9/54.7% over three hidden shuffles, and 26.3% at N=600 (0.17%
chance). First end-to-end world matching in the project: no pairs, no
declared dictionary, no learned cross-modal map -- factor families from
mutual exclusivity, colour correspondence from marginal frequency rank,
object states from pixels, assignment from a polynomial spectral solver
on two independently built relation fields.
Composing the two solvers closes most of the remaining gap: GRAMPA
initialisation 53.5%, plus pairwise-energy steepest descent 86.7%, plus
pairwise-and-triangle refinement **95.3%**. Spectral never gets trapped
but rounds badly; the energy rounds well but cannot find the basin; the
composition does both. The final state sits 0.04% below the truth in
energy at 95.3% accuracy -- residual field noise, not a counterfeit.
The theoretical account held throughout: nothing worked below rho = 0.9
and recovery appeared as soon as the fields crossed it. Reporting-bias
worlds show the derivation degrades gracefully -- text-versus-pixel rank
correlation 1.00/0.98/0.84/0.77 gives dictionaries of 8/8/6/6 out of ten
-- with errors concentrated in the close-frequency tail, so
margin-calibrated entries are the natural seed set.
The recovered assignment then transfers: used as pseudo-pairs for a ridge
map and applied to 200 held-out scenes that took no part in matching,
retrieval is 93.0% exact against 0.5% chance, colour set F1 0.989, count
multiset 97.5%. The random-pair control sits exactly at chance and the
shuffled-image control drops colour F1 to 0.295, so the output is
image-conditioned and extrapolates. Rungs four and five of the evidence
ladder are closed inside the synthetic world; matching is a bootstrap for
the map, not the deployment mechanism.
Unchanged by this: object states are hand-built upper-bound descriptors
that no tested self-supervised recipe reproduces, and the world is
procedurally generated.
See `SYNTH_RESULTS.md`.
## 2026-08-01: reporting bias, and three solver-versus-information calls
Worlds v3 and v4 test whether the derived dictionary survives reporting
bias. v3 couples colour to shape cyclically and biases mentions toward
rare colours, so captions name half the groups and the pixel and text
colour rankings diverge in the tail; v4 replaces the cyclic coupling with
a distinct Dirichlet shape profile per colour.
v3 results: marginal rank 4/10, joint structure 6/10, joint plus marginal
under exact enumeration of the 720 shape permutations 7/10, oracle shape
columns with measured clusters 8/10, oracle labels 10/10. Vision-side
extraction is not the limit (colour cluster purity 1.000, shape 0.998).
The 6/10 initially looked like an information bound -- ten colours over
six shapes leaves exactly four twin profiles, and exactly four appeared
-- but the twins separate by frequency, and the combined evidence has
minimum pairwise distance 0.2322 against 0.0008 for shape alone. The
bound is 10/10; what is left is one entry of solver gap and two of
estimation noise from the thinned joint table.
v4 results: exact enumeration recovers 10/10 under the same bias and
mention rate, across frequency weights including zero. Non-degenerate
joint structure suffices on its own, which is the regime real corpora
occupy.
Three times now a limit has looked intrinsic and proved to be the
solver's: descent retention, the triangle-energy pass, and this ceiling.
The rule stands -- compute the bound, do not infer it.
See `SYNTH_RESULTS.md`.
## 2026-08-01: factor families on real language
First step of the port to natural data, text side only, on 80,000 Visual
Genome region descriptions with no lexicon and no labels.
The synthetic pipeline found families by mutual exclusivity: template
phrases carry exactly one colour word, so colour terms never co-occur.
That does not transfer. In real descriptions colour words are
statistically independent or better -- observed over expected
co-occurrence is 1.29 for black with white, 0.93 for red with blue --
while head nouns are the genuinely exclusive family (0.14 for car with
truck). Run as written, the method recovers clean noun classes and no
attribute families at all. Mutual exclusivity is a template artifact.
Standard distributional word-class induction, of which exclusivity is a
degenerate case, does transfer. Positive PMI context vectors, truncated
SVD, k-means over the 300 most frequent words yields a colour family at
0.67 purity (red, brown, gray, dark, grey, pink, tan, purple) plus
coherent object categories: vehicles, animals, clothing, people, water
scenes. Colour terms split into three clusters rather than one, and the
split is structural rather than noisy -- white and blue cluster with sky
and clouds, because what a colour word modifies shapes its distribution.
That dependence is the signal cross-modal value alignment feeds on.
Next: the vision side under unsupervised segmentation, then the field
correlation as the go/no-go before any recovery run.
## 2026-08-01: first natural-data coordinate, and a correction
The Tier 0 recipe was ported to Visual Genome: DINOv2 patch features,
spectral segmentation of the affinity graph, segment feature classes on
one side; distributional word classes on the other. Field correlation at
the hidden pairing is **0.166**, far under the 0.9 that recovery needs,
so by our own go/no-go no recovery run is warranted yet.
A correction came with it. Aligning vision classes to text classes by
frequency rank and by an oracle give bit-identical fields -- 0.1662 both
ways -- while agreeing on 0 of 23 classes. Moment fields are built from
within-modality inner products, and permuting feature coordinates
consistently leaves every inner product unchanged, so relation fields are
gauge-invariant in the class labels and the dictionary never enters them.
The same was true unnoticed in the synthetic world, where oracle and
derived dictionaries produced identical fields. The dictionary is
required for encoding a new image into text coordinates, not for
matching; matching has always run on the fields alone.
The 0.166 therefore reads as representation quality, and localises the
deficit at vision-side granularity: six segments per image, background
included, against sixteen phrases about specific objects. A sixteen-
segment run is measuring whether granularity is the binding constraint.
## 2026-08-01: natural data reaches 0.656, and segmentation was not the wall
Full port to Visual Genome, measured by the go/no-go statistic. Three
results overturned expectations, each checked rather than inferred.
Discretisation was the dominant loss: hard class codes give 0.166,
continuous states 0.489. Relation fields need scene similarity only, so
quantising states first is pure attrition; the synthetic world hid this
because its states are discrete by construction.
Segmentation is not the bottleneck on photographs. Oracle region boxes
buy 0.061 over unsupervised spectral segmentation of DINOv2 patches
(0.550 against 0.489), the inverse of the synthetic world where
segmentation was the whole gap. Flat untextured scenes are adversarial
for object discovery in a way photographs are not.
Content projection is again the largest lever: 0.489 to **0.656**,
matching its earlier role at node level (0.21 to 0.61). Protocol-legal
and oracle-box configurations land within 0.007, so the residual is not
about object localisation.
Control: frozen general-purpose encoders underperform corpus-fitted
statistics (0.19 for DINOv2 crop CLS against Qwen phrase states), so
in-context features and corpus-specific distributional vectors are the
better substrate.
Augmentation orbits were then tried and give nothing: four random resized
crops per image, segmented independently with fields averaged, move the
correlation from 0.6559 to 0.6559. Re-renders in the closed world
resample layout, which is nuisance by construction; random crops perturb
framing, which self-supervised patch features already absorb, so the
average has nothing to cancel. Multi-view observation means separate
photographs, not transformations of one.
Verdict: 0.656 against the 0.9 recovery needs; no recovery run, and the
orbit lever is unavailable on this corpus.
See `NATURAL_RESULTS.md`.
## 2026-08-01: landscape reshaping does not substitute for correlation
At the natural-data field quality of 0.656, every solver returns chance.
Full-rank spectral initialisation gives 0.0039 against 0.0039 chance,
refinement 0.0078, and a rank-truncation ladder from four to full rank
stays at or below chance throughout. Coarse-to-sharp smoothing is the
standard way to widen basins and thin decoys, and it recovers nothing
here.
The reading is categorical rather than about this one method. Below the
polynomial threshold the deficit is information the algorithm class
cannot exploit, not a basin it fails to locate, so reshaping has nothing
to work with. Solver-side candidates should therefore wait on the
correlation rather than compete with it, and the only non-correlation
lever still open is population size, since the phase boundary is joint
in the two.
Compute note: NCSA Delta A40 was brought in for the correlation-raising
battery -- larger backbones and finer segmentation over the same corpus
-- with three array tasks starting immediately against a queue of 810,
9,624 GPU-hours remaining on the account.
## Current experimental gate
The node-level ordering-and-basin gate is passed on VG by content-projected
states (isolated and in-context composite). The standing rules extend
rather than retire:
1. Blind recovery next: unpaired soft-coupling optimization (Sinkhorn over
real states, preserving the decodable path) must place calibrated
coupling mass on the hidden truth from scratch. Its score is coupling
mass at truth and verified seed precision, never retrieval means alone.
2. The view-level gate stands unchanged for the coarse-to-fine recursion:
static view fields failed it; in-context view states plus projection
are the next candidate, with replayed view truth evaluation-only.
3. Any new energy or state family still passes ordering before descent,
including single-transposition local ordering.
4. Flickr-style single-view data cannot host the current pass; two-sided
multi-view observation is a protocol requirement for now.
|