World Alignment: unpaired cross-modal correspondence by relational identifiability

Method: scene states are sets of part states; relation fields are built
within each modality and are invariant to how each side labels its own
features; the cross-modal bridge is a coupling searched under an energy
that is a closed-form functional of one matrix; solving is spectral
initialisation followed by exact local refinement.

Evidence: in a procedurally generated closed world, blind recovery of a
hidden image-caption correspondence reaches 95.3% at 256 scenes against
0.39% chance, and the recovered pairs transfer to 200 held-out scenes at
93.0% exact retrieval with random-pair and shuffled-image controls at or
near chance. Cross-modal value correspondence is derived from disjoint
corpora rather than declared. On Visual Genome the field correlation
reaches 0.656 against the 0.9 that polynomial recovery needs, with the
deficit attributed away from segmentation and discretisation.

Protocol: no image-text pair enters any objective, optimiser,
initialisation, or model selection; hidden pairs score orderings only.

Co-Authored-By: Claude <noreply@anthropic.com>

1 files changed, 375 insertions, 0 deletions

diff --git a/artifacts/energy_functional_val_f30.json b/artifacts/energy_functional_val_f30.json
new file mode 100644
index 0000000..8e99c38
--- /dev/null
+++ b/artifacts/energy_functional_val_f30.json
@@ -0,0 +1,375 @@
+{
+  "protocol": "No image-text pair and no cross-modal parameter is used. The functional language energy is a frozen decode-reencode cycle through a text-only prefix interface and frozen Qwen.",
+  "mode": "functional_cycle_language_latent_particles",
+  "split": "val",
+  "rows": [
+    67,
+    99,
+    100,
+    109,
+    118,
+    161,
+    170,
+    200,
+    203,
+    257,
+    280,
+    282,
+    293,
+    301,
+    322,
+    370,
+    377,
+    382,
+    387,
+    390,
+    419,
+    422,
+    462,
+    478,
+    511,
+    608,
+    610,
+    628,
+    630,
+    669,
+    681,
+    694,
+    762,
+    816,
+    843,
+    900,
+    949,
+    1052,
+    1074,
+    1081,
+    1089,
+    1102,
+    1103,
+    1107,
+    1144,
+    1145,
+    1155,
+    1162,
+    1168,
+    1182,
+    1209,
+    1212,
+    1271,
+    1286,
+    1293,
+    1351,
+    1365,
+    1371,
+    1378,
+    1382,
+    1478,
+    1513,
+    1516,
+    1520,
+    1523,
+    1532,
+    1616,
+    1636,
+    1716,
+    1727,
+    1740,
+    1746,
+    1754,
+    1782,
+    1892,
+    1902,
+    1933,
+    1943,
+    1995,
+    2006,
+    2089,
+    2145,
+    2147,
+    2177,
+    2179,
+    2256,
+    2274,
+    2283,
+    2312,
+    2324,
+    2326,
+    2388,
+    2434,
+    2476,
+    2522,
+    2527,
+    2605,
+    2662,
+    2737,
+    2749,
+    2968,
+    3032,
+    3085,
+    3096,
+    3104,
+    3105,
+    3115,
+    3175,
+    3180,
+    3197,
+    3203,
+    3204,
+    3304,
+    3375,
+    3393,
+    3410,
+    3423,
+    3425,
+    3434,
+    3461,
+    3493,
+    3498,
+    3510,
+    3529,
+    3547,
+    3566,
+    3631,
+    3681
+  ],
+  "args": {
+    "manifest": "artifacts/manifest.json",
+    "vision": "artifacts/vision.pt",
+    "text": "artifacts/text.pt",
+    "text_orbits": "artifacts/text_orbits_qwen0p5b.pt",
+    "prefix": "artifacts/prefix.pt",
+    "split": "val",
+    "samples": 128,
+    "steps": 60,
+    "refresh_steps": 20,
+    "max_new_tokens": 24,
+    "lr": 0.03,
+    "projections": 128,
+    "relation_weight": 1.0,
+    "conditional_weight": 0.08,
+    "distribution_weight": 80.0,
+    "functional_weight": 30.0,
+    "device": "cuda:1",
+    "seed": 20260729,
+    "output": "artifacts/energy_functional_val_f30.pt",
+    "metrics_output": "artifacts/energy_functional_val_f30.json"
+  },
+  "history": [
+    {
+      "step": 0,
+      "total": 3.3505666255950928,
+      "relation": 2.1214420795440674,
+      "conditional": 5.643650531768799,
+      "distribution": 4.243086550559383e-06,
+      "functional": 0.02590976282954216,
+      "paired_evaluation_only": {
+        "r@1": 0.015625,
+        "r@5": 0.0390625,
+        "r@10": 0.09375,
+        "median_rank": 57.0,
+        "paired_cosine": 0.9550588130950928
+      }
+    },
+    {
+      "step": 10,
+      "total": 2.516568660736084,
+      "relation": 0.6655093431472778,
+      "conditional": 3.2645981311798096,
+      "distribution": 4.488533522817306e-05,
+      "functional": 0.05287669226527214,
+      "paired_evaluation_only": {
+        "r@1": 0.0,
+        "r@5": 0.046875,
+        "r@10": 0.1171875,
+        "median_rank": 52.0,
+        "paired_cosine": 0.8903882503509521
+      }
+    },
+    {
+      "step": 20,
+      "total": 2.5986273288726807,
+      "relation": 0.6709409952163696,
+      "conditional": 3.5192151069641113,
+      "distribution": 2.1296296836226247e-05,
+      "functional": 0.05481484532356262,
+      "paired_evaluation_only": {
+        "r@1": 0.015625,
+        "r@5": 0.0859375,
+        "r@10": 0.1328125,
+        "median_rank": 53.0,
+        "paired_cosine": 0.9171137809753418
+      }
+    },
+    {
+      "step": 30,
+      "total": 1.4995677471160889,
+      "relation": 0.5183121562004089,
+      "conditional": 3.07732892036438,
+      "distribution": 2.971702633658424e-05,
+      "functional": 0.024423060938715935,
+      "paired_evaluation_only": {
+        "r@1": 0.0078125,
+        "r@5": 0.0390625,
+        "r@10": 0.078125,
+        "median_rank": 57.0,
+        "paired_cosine": 0.9059231281280518
+      }
+    },
+    {
+      "step": 40,
+      "total": 2.2073776721954346,
+      "relation": 0.5520342588424683,
+      "conditional": 3.3222787380218506,
+      "distribution": 2.083627987303771e-05,
+      "functional": 0.04626314342021942,
+      "paired_evaluation_only": {
+        "r@1": 0.0078125,
+        "r@5": 0.0546875,
+        "r@10": 0.09375,
+        "median_rank": 54.0,
+        "paired_cosine": 0.9188947677612305
+      }
+    },
+    {
+      "step": 50,
+      "total": 1.3839813470840454,
+      "relation": 0.4820159077644348,
+      "conditional": 3.0760395526885986,
+      "distribution": 2.733840847213287e-05,
+      "functional": 0.02178984135389328,
+      "paired_evaluation_only": {
+        "r@1": 0.015625,
+        "r@5": 0.0625,
+        "r@10": 0.125,
+        "median_rank": 56.0,
+        "paired_cosine": 0.9088481068611145
+      }
+    },
+    {
+      "step": 60,
+      "total": 1.9196703433990479,
+      "relation": 0.49436357617378235,
+      "conditional": 3.271376371383667,
+      "distribution": 2.1426760213216767e-05,
+      "functional": 0.03872941434383392,
+      "paired_evaluation_only": {
+        "r@1": 0.0234375,
+        "r@5": 0.0625,
+        "r@10": 0.1015625,
+        "median_rank": 57.0,
+        "paired_cosine": 0.9172194600105286
+      }
+    }
+  ],
+  "prefix_training": "text only; no image features or image-text pairs",
+  "refresh_caption_examples": {
+    "0": [
+      "Two women making pasta with large amounts of black rice. 2009. 001. 0",
+      "A toddler is playing with a toy in a beach pool. The boy is holding a toy.",
+      "Three men in a conference room talk to an interviewer in a white shirt. 0.000000",
+      "A group of men are eating at a table with a large pizza. They are all wearing white shirts.",
+      "Two girls look at a woman wearing a pink shirt. She is holding a camera. 0 1 2 ",
+      "A brown dog runs through the snow with his tongue out. Supervised by a human.  License: CC-BY",
+      "Two women dressed in black and white pants and headscarves are leaning against a wall. \nAssociation: \"The Art",
+      "A bike rider is doing a trick on his bicycle in the air. He is balancing on his bike. \nAttachment:",
+      "A young man in a business suit holds his cellphone.  He is talking to someone.  He is in a cafe",
+      "A man and woman kissing on a pink convertible with a red and white car behind.  Near a subway station. ",
+      "A man sits on a chair holding a rope while standing in the middle of a rocky beach.",
+      "A couple, one with a white shirt and the other with a black jacket, are going out on a sunny day.",
+      "A blond child in a red hat sits inside a cardboard box.",
+      "A man in black glasses and blue shirt looks to the side of a rock.",
+      "A man playing drums in a square playing a black hat.  He is singing.  The crowd is watching. ",
+      "Two young men skateboarding in the street in the background.  His skateboarded on the street.  License plate is",
+      "A soldier with a gold vest and military uniform is wearing a safety vest. He is holding a gun.",
+      "A large dog and two small dogs lying on the grass in a dense forest. Large dog and small dog sit together.",
+      "A black dog is chasing a large brown bull. The dog is close to the bull.",
+      "A baseball player slides as he attempts to catch a ball as the opposing team's player slides to the ground. Caption:",
+      "A young boy is swinging a baseball bat. He is playing with a baseball.",
+      "A guy wearing a clean shirt is sweeping the floor. He's cleaning the windows.",
+      "A girl wearing a blue shirt is riding in a shopping cart on the inside of a bus.",
+      "Three people are setting up chairs on a beach with a tent.",
+      "A man paints a mural on a painted wall in a busy street."
+    ],
+    "20": [
+      "Two men in a large pasta making, 20% off.  Image of 2000's ",
+      "A young man is playing with a beach toy in a swimming pool, while a toddler is holding a bottle.",
+      "One man in a baseball cap and a large bookless frame speaks to a group of young men. 旁观者",
+      "A group of people are eating a large pizza at a restaurant, all wearing white shirts. One man is eating alone.",
+      "Two girls are holding a picture of a woman and a child.",
+      "A child in a yellow jacket catches a ball with his hand. Title: Golden Retriever Enjoying the Ball.",
+      "Two people are wearing black skirts and are holding up a sign. \n#Fictional place: a city street.",
+      "A bike racer is doing a trick on his bike, and is in the air. The picture is taken from the side",
+      "A young man in a business suit, has his hands in his pockets, while holding a cellphone in his hand. He",
+      "A man and a woman kissing on a red car, with a pink and white car in the background. The car is",
+      "A man stands in the middle of a rope悬挂的水面, holding a rope. \nA:\nAssistant: 选项",
+      "A man and a woman, both in white, are outside, with a lake in the background.",
+      "A red-haired blond dog in a baseball cap sits in a chair. He is wearing a green shirt.",
+      "A young man on a wooden bench looks at a large photograph of a mountain. Behind him is a large red truck.",
+      "A man is playing a band, and is standing in a square, and the floor is covered with a band. He",
+      "Two men skate on the street in black and white, riding bikes.  They look cool.  They are skateboarding",
+      "A military man with a military uniform is carrying a pistol and wearing a seat belt.",
+      "One dog and several other dogs outside a small house. People walk around a field.",
+      "A small dog is running with the strong man, while the blue sky is behind them.",
+      "A baseball player slides down the field as the ball is thrown and a car passes by in the background.",
+      "A young boy is swinging on a playground swing. He is wearing a blue baseball cap. This is likely a youth event",
+      "The man is washing the floor with a white shirt. Cleaning the floor.",
+      "A boy is riding a blue shirt in front of a boat on a beach while drinking from a bottle.",
+      "Three people are sitting on a beach with a tent and a motor boat.",
+      "A man paints a mural on a glass panel in a room with a blue wall. The room is decorated with a lot"
+    ],
+    "40": [
+      "2 small, 2 male, 2 half of 50's, 2 in front of 1 large",
+      "A young boy is playing in a pool with his hand in a swimming pool, while a woman holds a ball in her",
+      "One man with a baseball cap and a large brown ball fielded a ball to a group of children and a large group",
+      "A group of people, all dressed in red, are eating at a table.",
+      "A woman and a girl are holding a picture of a snowboarder.",
+      "A boy in a green jacket catches a baseball with a yellow ball in the field.",
+      "A person is wearing a pair of pants and a sign. \nQ: What is the name of the person wearing a",
+      "The bike that is on the ground is doing a flip and is in the air. The person on the bike is wearing",
+      "A young man in a suit, his hand in his pocket, has his hand in his wallet while holding his hand in",
+      "A man and a woman kissing on a red car with a pink and white car in the background.",
+      "A man stands in the water, holding a rope.  He is swimming.  He is holding a rope. ",
+      "A man and a woman are in the park, with blue and white walls. \nFictional Scene: A man",
+      "A red-colored dog in a sweater and a brown hat sits in the grass. The dog looks happy.",
+      "A young man looks at a large, red, wooden car on a wooden bench in front of a large, snowy mountain",
+      "A man is playing a band, and a space is filled with various instruments. The man is also standing, and some",
+      "Two men are skating, both in black and white, on skateboards. They are in a park. Caption: Skate",
+      "A military member is wearing a military uniform with a military gear. He has a military gear with a military insignia on",
+      "Several dogs walking outside of a large building with a tree in the background. People walk by.",
+      "The small dog is the main focus, the young woman is running with the big blue ball. Large crowd of people behind",
+      "A young baseball player slides down a baseball field as a red car passes by with a baseball in the background.",
+      "A young boy is swinging on a playground swing, wearing a blue shirt and a white hat. The swing is near a",
+      "The man washing the floor with a white towel. Cleaning floor. Cleaning floor. Cleaning floor.",
+      "A boy is drinking from a cup in front of a boat while riding a bike in the water with a flag on his",
+      "Three people are sitting on a small beach with a portable table and a car.",
+      "A man is painting a room with a large window in it. The room is painted in a bright color. The man"
+    ],
+    "60": [
+      "20+  male, 20+  60s, 60 min, 2x",
+      "A young boy is in a swimming pool with his eyes fixed on a young girl in a swimming pool. He is holding",
+      "Several children with a large ball and a man in a blue shirt and a large brown dog are running on a field with",
+      "A man, dressed in red, blue, and green, is eating at a table.",
+      "Two women and a child are holding a snow leopard.",
+      "A boy in a baseball cap catches a baseball in a field with a green ball in his hand.",
+      "A person is wearing a pair of pants and a hat.  A sign is also on the floor.  A sign",
+      "The bike on the ground is doing a flip in the air, on a path.",
+      "A young man in a suit has his hand in his pocket, while holding his hand in his other hand. He has",
+      "A man and a woman kissing with a red car in the background.",
+      "The man is swimming, holding a rope. , in the water. \n\n###### Line 1:  A man in",
+      "A man and a woman are in a park, in blue and white, with flowers in the background. \nISBN:",
+      "Red dog in a red sweater sitting in a field. Red dog in a red sweater sits in a field. (photo",
+      "A middle-aged man looks at a large, red, heavily-laden, old car near a wooden, unpainted,",
+      "A man is playing a musical instrument, while a carpet is being spread, and a white space is in the background.",
+      "Two men are skating in a park, in the style of \"The Wrestlers\".  The men are both in black",
+      "A military member has a uniform with a military badge and is wearing a military gear with a military member with a military hat",
+      "Several dogs walking in a park with a tree in the background and a house in the distance.",
+      "The big dog is the most tired out of the group of people, but the small baby is the most active. Geographic",
+      "A young boy runs past a baseball player with a red car on the field as it passes by him in the background.",
+      "A young boy is swinging on a swing at a playground with a blue and white umbrella. The boy is wearing a blue",
+      "Woman sweeping the floor with a white towel on the floor. Cleaning the house.",
+      "A boy is drinking from a can in front of a boat with a person in the water in the background with a boat",
+      "Three people are sitting on a small beach with a large portable table and a small car.",
+      "A man is painting a room with a large window. The room is painted in a dark color. The man is wearing"
+    ]
+  }
+}
\ No newline at end of file