path: root/MANIFOLD_RESULTS.md

# On-manifold assignment gate

Run date: 2026-07-29

## Question

The free-particle energy was falsified off-manifold: optimization left the
decodable language region and produced counterfeit states below the true
configuration. This experiment removes that escape route by construction.
Configurations are permutations of real frozen text states, so the
language-only terms of the falsified energy (sliced distribution, prototype
manifold) are constants, and the only varying terms are the cross-modal
relation MSE and conditional KL. If the earlier failure was caused by
leaving the manifold, the true assignment should now be an energy minimum.

It is not. The falsification survives the restriction, and the gate
localizes the cause in the states rather than the search space.

## Protocol

- Configuration space: permutations of held-out real text states. Hidden
  pairs place the true assignment in the ranking and are never visible to
  the energy.
- Gate A, global ordering: energy of the true assignment against 1,000
  uniform permutations and a similarity-ordered cyclic shift.
- Gate B, local ordering: the exact energy change of every transposition of
  the true assignment. The quadratic text terms cancel, so one matrix
  product evaluates all N(N-1)/2 swaps; the closed form is unit-tested
  against brute-force recomputation.
- Gate C, basin audit: exact steepest 2-swap descent from the true
  assignment and from random assignments, with a local-minimum certificate.
  Random-start descent doubles as a search for on-manifold counterfeits.
- Conditions: Flickr30k test/val (DINOv2-small CLS against Qwen2.5-0.5B
  mean-pooled states, single caption and five-caption orbit means) and
  Visual Genome 512-node subsets and the full 5,000 nodes (16-view means,
  `region_closed` and `visible_relations` tiers, scalar and
  distribution-valued relation channels).

## The ordering passes globally and fails locally

Every condition separates the true assignment from random permutations;
no condition makes it a local minimum.

| Condition | Random-perm z | Improving swaps | Kept by descent | E true | E counterfeit |
|---|---:|---:|---:|---:|---:|
| Flickr test 512, orbit | 18.7 | 7.4% | 13.9% | 4.17 | 3.26 |
| Flickr test 512, single | 10.8 | 15.6% | 6.6% | 4.67 | 3.44 |
| Flickr val 512, orbit | 18.0 | 6.8% | 11.9% | 4.10 | 3.26 |
| Flickr test 1,000, orbit | 25.3 | 7.3% | 11.0% | 4.30 | 3.42 |
| VG 512, region_closed | 4.9 | 23.5% | 2.7% | 4.58 | 2.73 |
| VG 512, visible_relations | 9.2 | 17.9% | 3.7% | 4.14 | 2.27 |
| VG 512, region bundle | 2.7 | 33.2% | 4.5% | 4.86 | 3.22 |
| VG 5,000, region_closed | 18.3 | 21.6% | (descent disabled) | 4.69 | -- |
| VG 5,000, visible_relations | 28.2 | 17.7% | (descent disabled) | 4.39 | -- |

"Improving swaps" is the fraction of all transpositions of the true
assignment that lower the energy: 9,723 of 130,816 in the best condition,
2,697,600 of 12,497,500 at N=5,000. Scale does not repair local ordering.

VG rows show subset seed 0; seeds 1 and 2 replicate every verdict. Across
the twelve VG subset runs (3 seeds x 2 tiers x 2 channel modes), improving
swaps span 15.7--33.2%, descent retention 2.7--22.5%, and a counterfeit
below the true energy by 1.6--2.2 is found in 12 of 12 runs.

The derangement curve shows where the energy has resolution. On Flickr
orbit, permutations that displace 32 or more nodes are detected in 200 of
200 samples; transpositions are missed in 9% of samples. On VG the same
inversion holds (k=2: 18%, k=128: 0%). **The energy resolves coarse
rearrangements and is blind to fine ones.**

## Descent manufactures counterfeits from every start

Steepest descent from the true assignment walks away from it in every
condition, keeping 2.7--13.9% of nodes. Quenches from random assignments
reach energies 0.9--1.9 below the true configuration at 0.0--0.4% accuracy.
The counterfeit family is not an artifact of continuous relaxation; it is a
property of the assignment landscape itself:

E(true) > E(wrong local minimum), for every observed minimum.

Five-caption orbits halve the improving-swap fraction (15.6% to 7.4%) and
double descent retention (6.6% to 13.9%); explicitly visible relations cut
it by a quarter (23.5% to 17.9%). Both help, neither changes the verdict.

## The mechanism is spectral: about sixteen shared dimensions

Three measurements locate the failure in the node states.

First, the pooled text states are extremely anisotropic: 99.5% of all
Flickr caption pairs have cosine above 0.8, and every VG
`visible_relations` pair is above 0.9. Raw cosine carries almost no
instance identity; the energy operates on standardized relation values,
whose cross-modal Spearman correlation is 0.19 (single caption), 0.31
(orbit mean), 0.21 (`region_closed`), 0.30 (`visible_relations`). Roughly
90% of fine relational variance is modality-private.

Second, the shared part is spectrally concentrated. Projecting both
modalities onto their top population principal components before computing
relations gives:

| Kept dimensions | Cross-modal Spearman |
|---:|---:|
| 2 | 0.315 |
| 16 | 0.367 |
| 128 | 0.322 |
| 384 | 0.315 |
| untruncated | 0.306 |

Two directions already carry more cross-modal relational signal than the
full representation; the optimum near sixteen dimensions gains 0.06 over
full; every dimension past that dilutes. Population whitening, which
equalizes the spectrum, collapses the correlation to 0.03. **The shared
world signal lives in a roughly sixteen-dimensional coarse subspace, and
the fine structure that identification needs is modality-private.**

Third, the worst improving swaps are pairs whose text states are nearly
parallel while their scenes are unrelated: text cosine 0.88--0.94 against
visual cosine -0.13--0.15. Example pair at text cosine 0.897: "A baseball
player in red leaps into the air..." against "A woman in a blue shirt is
walking across the street...". A sixteen-dimensional topical code cannot
separate 512 instances, and no functional of these relation fields can.

## Higher moments and view bundles buy nothing at node level

The distribution-valued relation channels (std, q10, q90 of the 256
view-pair cosines between two VG nodes) were a direct test of richer edge
structure. Their cross-modal Spearman is -0.02, 0.05, and 0.11--0.16
respectively, against 0.21--0.30 for the mean channel, and they are
strongly entangled with it within each modality (rank correlation -0.82 to
0.63). Adding them raises the improving-swap fraction from 23.5% to 33.2%:
channels without shared signal act as noise. Model scale was already known
to be inert here (Qwen 0.5B to 1.5B left relational diagnostics unchanged),
and whitening is destructive, so the node-state toolbox is exhausted:
every cheap transform either denoises within the sixteen shared dimensions
or dilutes them. None expands them.

## Region-level structure inside nodes is shared

The constructive result. Both VG modalities observe the same sixteen
regions per scene; the preparation RNG that hid the view correspondence is
replayable from the seed and the private image IDs, and the replay
reconstructs all 5,000 released phrase bundles exactly, so view-level truth
is available for evaluation.

On all 5,000 nodes:

- Within-node relation fields align: Spearman between the visual and
  truth-aligned text view-relation fields is 0.128 against -0.002 for
  shuffled views, z = 46.6 across nodes.
- Views become matchable through the coarse frame: ranking each view's
  relation profile against a 1,000-node population frame and solving the
  16-view assignment gives 14.5% accuracy against 6.25% chance (2.3x;
  top-3 29.5% against 18.8%). Growing the frame from 200 to 1,000 nodes
  raises accuracy from 12.9% to 14.5%. This uses the true node alignment
  as the frame, so it is an upper bound on what a recovered coarse
  alignment would enable.
- Within-node relations alone reach 7.0% on a 200-node sample: the
  population context, not the 16-view geometry, carries the signal.

The fine-grained shared structure the node-level energy lacks does exist
one level down, in the compositional structure the pooled states discard.

## Ricci flow doubles the frame and fails the resolution

A discrete Ollivier-Ricci flow was run as a control for the hypothesis
that smoothing each modality's relational geometry could repair local
ordering. Each modality's kNN graph evolved independently for ten
iterations; identical heat-kernel channels were gated with and without the
flow, so the flow's own effect is isolated.

| Condition | Random-perm z | Improving swaps | Kept by descent | E true | E counterfeit |
|---|---:|---:|---:|---:|---:|
| Flickr orbit 512, diffusion only | 21.4 | 15.2% | 4.1% | 1.263 | 0.717 |
| Flickr orbit 512, Ricci flow | 47.8 | 9.1% | 5.1% | 0.818 | 0.427 |
| VG 512 region, diffusion only | 6.9 | 22.9% | 3.1% | 1.601 | 0.648 |
| VG 512 region, Ricci flow | 22.5 | 13.9% | 1.2% | 1.405 | 0.650 |

The flow is a genuinely good coarse-channel denoiser: it more than
doubles the global separation in both datasets. It changes no verdict:
9--14% of transpositions still improve the flowed energy, descent keeps
1--5% of the truth, and every random quench finds a counterfeit at about
half the true energy. **Better geometry buys a better frame, not a better
resolver** -- the two roles dissociate exactly as the spectral audit
predicted, so geometric flows belong in the coarse-frame layer of the next
design and cannot substitute for finer states.

## Static view-level energies fail the same gate

The view-level continuation gates the structured states directly. Given
the true coarse frame (upper-bound scaffolding), the true view assignment
is a strict local minimum for none of 5,000 nodes: the within-node
relational energy has 32.0% improving view-transpositions, the
population-profile energy 28.3%, and their best combination 25.5%.
Frame-free matching of internal relation fields (signature-initialized
Hungarian plus exact batched 2-swap descent, with a value-scrambled
capacity null and column centering) is dead at 512 nodes across three
seeds: R@10 2.0--3.3% against 2.0% chance, Hungarian recovery at most
0.6%, top-margin-decile precision 0 of 51, and the true-pair cost
separation is z = 0.1--1.8. A small negative separation seen at the
96-node smoke scale did not replicate and is recorded as noise.

Static within-node relation fields therefore carry shared signal (the
z = 46.6 alignment result) but not enough for identification even at
sixteen candidates -- the view-level gate needs model-internal relations
(joint-context attention) or true re-encoding interventions, which is the
R3/R8 battery of `STRUCTURE_DESIGN.md`.

## Conclusion

The manifold restriction was necessary and is now insufficient for a
different reason than the free-particle failure. The energy is not being
cheated by off-manifold vectors; it is blind at the resolution where
instance identity lives, because pooled node states expose only a coarse
shared subspace. No matcher, objective reweighting, or scale change fixes
an information ceiling in the states.

The gate for the next experiment therefore moves down one level:

1. a node state is a set of region views with its internal relation field,
   not a pooled vector;
2. the candidate energy scores view-level assignments conditioned on a
   coarse frame, and must first pass this same transposition/descent gate
   at the view level, with the replayed view truth as hidden evaluation;
3. response operators under controlled interventions (view removal,
   attribute edits) are the candidate source of shared fine structure
   beyond static view relations, to be tested through the identical gate
   before any descent run.

## Main artifacts

- `artifacts/manifold_gate/flickr_test512_{orbit,single}.json`
- `artifacts/manifold_gate/flickr_val512_orbit.json`
- `artifacts/manifold_gate/flickr_test1000_orbit.json`
- `artifacts/manifold_gate/vg512_{region_closed,visible_relations}_seed{0,1,2}_{scalar,bundle}.json`
- `artifacts/manifold_gate/vg5000_{region_closed,visible_relations}_scalar.json`
- `artifacts/manifold_gate/resolution_audit.json`
- `artifacts/manifold_gate/spectral_localization.json`
- `artifacts/manifold_gate/bundle_channel_audit.json`
- `artifacts/manifold_gate/vg_view_probe_smoke.json`
- `artifacts/manifold_gate/vg_view_probe_full.json`
- `artifacts/manifold_gate/ricci_flickr_test512_orbit.json`
- `artifacts/manifold_gate/ricci_vg512_region_seed0.json`
- `artifacts/manifold_gate/view_gate_full.json`
- `artifacts/manifold_gate/view_gate_cost_seed{0,1,2}.pt`