World Alignment: unpaired cross-modal correspondence by relational identifiability

Method: scene states are sets of part states; relation fields are built
within each modality and are invariant to how each side labels its own
features; the cross-modal bridge is a coupling searched under an energy
that is a closed-form functional of one matrix; solving is spectral
initialisation followed by exact local refinement.

Evidence: in a procedurally generated closed world, blind recovery of a
hidden image-caption correspondence reaches 95.3% at 256 scenes against
0.39% chance, and the recovered pairs transfer to 200 held-out scenes at
93.0% exact retrieval with random-pair and shuffled-image controls at or
near chance. Cross-modal value correspondence is derived from disjoint
corpora rather than declared. On Visual Genome the field correlation
reaches 0.656 against the 0.9 that polynomial recovery needs, with the
deficit attributed away from segmentation and discretisation.

Protocol: no image-text pair enters any objective, optimiser,
initialisation, or model selection; hidden pairs score orderings only.

Co-Authored-By: Claude <noreply@anthropic.com>

1 files changed, 375 insertions, 0 deletions

diff --git a/artifacts/energy_functional_val_f10.json b/artifacts/energy_functional_val_f10.json
new file mode 100644
index 0000000..a1ee13e
--- /dev/null
+++ b/artifacts/energy_functional_val_f10.json
@@ -0,0 +1,375 @@
+{
+  "protocol": "No image-text pair and no cross-modal parameter is used. The functional language energy is a frozen decode-reencode cycle through a text-only prefix interface and frozen Qwen.",
+  "mode": "functional_cycle_language_latent_particles",
+  "split": "val",
+  "rows": [
+    67,
+    99,
+    100,
+    109,
+    118,
+    161,
+    170,
+    200,
+    203,
+    257,
+    280,
+    282,
+    293,
+    301,
+    322,
+    370,
+    377,
+    382,
+    387,
+    390,
+    419,
+    422,
+    462,
+    478,
+    511,
+    608,
+    610,
+    628,
+    630,
+    669,
+    681,
+    694,
+    762,
+    816,
+    843,
+    900,
+    949,
+    1052,
+    1074,
+    1081,
+    1089,
+    1102,
+    1103,
+    1107,
+    1144,
+    1145,
+    1155,
+    1162,
+    1168,
+    1182,
+    1209,
+    1212,
+    1271,
+    1286,
+    1293,
+    1351,
+    1365,
+    1371,
+    1378,
+    1382,
+    1478,
+    1513,
+    1516,
+    1520,
+    1523,
+    1532,
+    1616,
+    1636,
+    1716,
+    1727,
+    1740,
+    1746,
+    1754,
+    1782,
+    1892,
+    1902,
+    1933,
+    1943,
+    1995,
+    2006,
+    2089,
+    2145,
+    2147,
+    2177,
+    2179,
+    2256,
+    2274,
+    2283,
+    2312,
+    2324,
+    2326,
+    2388,
+    2434,
+    2476,
+    2522,
+    2527,
+    2605,
+    2662,
+    2737,
+    2749,
+    2968,
+    3032,
+    3085,
+    3096,
+    3104,
+    3105,
+    3115,
+    3175,
+    3180,
+    3197,
+    3203,
+    3204,
+    3304,
+    3375,
+    3393,
+    3410,
+    3423,
+    3425,
+    3434,
+    3461,
+    3493,
+    3498,
+    3510,
+    3529,
+    3547,
+    3566,
+    3631,
+    3681
+  ],
+  "args": {
+    "manifest": "artifacts/manifest.json",
+    "vision": "artifacts/vision.pt",
+    "text": "artifacts/text.pt",
+    "text_orbits": "artifacts/text_orbits_qwen0p5b.pt",
+    "prefix": "artifacts/prefix.pt",
+    "split": "val",
+    "samples": 128,
+    "steps": 60,
+    "refresh_steps": 20,
+    "max_new_tokens": 24,
+    "lr": 0.03,
+    "projections": 128,
+    "relation_weight": 1.0,
+    "conditional_weight": 0.08,
+    "distribution_weight": 80.0,
+    "functional_weight": 10.0,
+    "device": "cuda:1",
+    "seed": 20260729,
+    "output": "artifacts/energy_functional_val_f10.pt",
+    "metrics_output": "artifacts/energy_functional_val_f10.json"
+  },
+  "history": [
+    {
+      "step": 0,
+      "total": 2.832371234893799,
+      "relation": 2.1214420795440674,
+      "conditional": 5.643650531768799,
+      "distribution": 4.243086550559383e-06,
+      "functional": 0.02590976282954216,
+      "paired_evaluation_only": {
+        "r@1": 0.015625,
+        "r@5": 0.0390625,
+        "r@10": 0.09375,
+        "median_rank": 57.0,
+        "paired_cosine": 0.9550588130950928
+      }
+    },
+    {
+      "step": 10,
+      "total": 1.8404440879821777,
+      "relation": 0.3791850507259369,
+      "conditional": 1.1169235706329346,
+      "distribution": 0.00012638189946301281,
+      "functional": 0.13617946207523346,
+      "paired_evaluation_only": {
+        "r@1": 0.0078125,
+        "r@5": 0.046875,
+        "r@10": 0.1328125,
+        "median_rank": 51.0,
+        "paired_cosine": 0.7836891412734985
+      }
+    },
+    {
+      "step": 20,
+      "total": 1.557417631149292,
+      "relation": 0.41428446769714355,
+      "conditional": 1.8413759469985962,
+      "distribution": 4.0761849959380925e-05,
+      "functional": 0.09925621002912521,
+      "paired_evaluation_only": {
+        "r@1": 0.015625,
+        "r@5": 0.0703125,
+        "r@10": 0.109375,
+        "median_rank": 50.0,
+        "paired_cosine": 0.8743711113929749
+      }
+    },
+    {
+      "step": 30,
+      "total": 0.9876586198806763,
+      "relation": 0.2846721112728119,
+      "conditional": 1.4240365028381348,
+      "distribution": 5.669375605066307e-05,
+      "functional": 0.05845280736684799,
+      "paired_evaluation_only": {
+        "r@1": 0.015625,
+        "r@5": 0.09375,
+        "r@10": 0.1953125,
+        "median_rank": 48.0,
+        "paired_cosine": 0.863619863986969
+      }
+    },
+    {
+      "step": 40,
+      "total": 1.4016644954681396,
+      "relation": 0.27223390340805054,
+      "conditional": 1.9623196125030518,
+      "distribution": 3.330874824314378e-05,
+      "functional": 0.09697803109884262,
+      "paired_evaluation_only": {
+        "r@1": 0.0234375,
+        "r@5": 0.109375,
+        "r@10": 0.1640625,
+        "median_rank": 49.0,
+        "paired_cosine": 0.8904920816421509
+      }
+    },
+    {
+      "step": 50,
+      "total": 1.0091907978057861,
+      "relation": 0.2396688014268875,
+      "conditional": 1.3726451396942139,
+      "distribution": 5.902868724660948e-05,
+      "functional": 0.06549880653619766,
+      "paired_evaluation_only": {
+        "r@1": 0.03125,
+        "r@5": 0.0859375,
+        "r@10": 0.1640625,
+        "median_rank": 43.0,
+        "paired_cosine": 0.8608105778694153
+      }
+    },
+    {
+      "step": 60,
+      "total": 1.2123510837554932,
+      "relation": 0.27834945917129517,
+      "conditional": 1.7881872653961182,
+      "distribution": 4.158633601036854e-05,
+      "functional": 0.0787619799375534,
+      "paired_evaluation_only": {
+        "r@1": 0.015625,
+        "r@5": 0.09375,
+        "r@10": 0.1796875,
+        "median_rank": 43.0,
+        "paired_cosine": 0.8811556696891785
+      }
+    }
+  ],
+  "prefix_training": "text only; no image features or image-text pairs",
+  "refresh_caption_examples": {
+    "0": [
+      "Two women making pasta with large amounts of black rice. 2009. 001. 0",
+      "A toddler is playing with a toy in a beach pool. The boy is holding a toy.",
+      "Three men in a conference room talk to an interviewer in a white shirt. 0.000000",
+      "A group of men are eating at a table with a large pizza. They are all wearing white shirts.",
+      "Two girls look at a woman wearing a pink shirt. She is holding a camera. 0 1 2 ",
+      "A brown dog runs through the snow with his tongue out. Supervised by a human.  License: CC-BY",
+      "Two women dressed in black and white pants and headscarves are leaning against a wall. \nAssociation: \"The Art",
+      "A bike rider is doing a trick on his bicycle in the air. He is balancing on his bike. \nAttachment:",
+      "A young man in a business suit holds his cellphone.  He is talking to someone.  He is in a cafe",
+      "A man and woman kissing on a pink convertible with a red and white car behind.  Near a subway station. ",
+      "A man sits on a chair holding a rope while standing in the middle of a rocky beach.",
+      "A couple, one with a white shirt and the other with a black jacket, are going out on a sunny day.",
+      "A blond child in a red hat sits inside a cardboard box.",
+      "A man in black glasses and blue shirt looks to the side of a rock.",
+      "A man playing drums in a square playing a black hat.  He is singing.  The crowd is watching. ",
+      "Two young men skateboarding in the street in the background.  His skateboarded on the street.  License plate is",
+      "A soldier with a gold vest and military uniform is wearing a safety vest. He is holding a gun.",
+      "A large dog and two small dogs lying on the grass in a dense forest. Large dog and small dog sit together.",
+      "A black dog is chasing a large brown bull. The dog is close to the bull.",
+      "A baseball player slides as he attempts to catch a ball as the opposing team's player slides to the ground. Caption:",
+      "A young boy is swinging a baseball bat. He is playing with a baseball.",
+      "A guy wearing a clean shirt is sweeping the floor. He's cleaning the windows.",
+      "A girl wearing a blue shirt is riding in a shopping cart on the inside of a bus.",
+      "Three people are setting up chairs on a beach with a tent.",
+      "A man paints a mural on a painted wall in a busy street."
+    ],
+    "20": [
+      "Two batches of 5x4 half-dimmed pasta, 2 people making.  Scale not given. ",
+      "A toddler is playing with a toy in a store, using a container. He is holding a toy gun. He is",
+      "An unassuming man in a green baseball uniform jumps into a field with a large, confident looking camera in his hand.",
+      "There is a large table with a lot of people eating a large cake. Some of the people are wearing white shirts.",
+      "Three girls are looking at a wooden structure. The children are wearing dresses. \n\nWhat is the name of the wooden structure",
+      "A child is seen jumping in front of a camera, while a man watches in the background.",
+      "Two people are wearing street clothes and are standing.",
+      "A bicyclist is doing a trick in the air, and is balancing on his bicycle.  A picture of a bicy",
+      "A man is in the office, talking on the phone. He is also holding his coffee. He is in the bathroom",
+      "Girl and Boy kissing on the dance floor with a red and white car in the background. Beaches and street lights are",
+      "A man stands in a harness while holding a rope. \n\nWhat is the correct answer to the question: What is the",
+      "A person with black and white, a kitchen with a window, outside.",
+      "A yellow dog sits in a chair with a green ball in his mouth. The dog has a red hat on his head",
+      "A man in white gloves looks at a few pieces of wood.  His face obscured.  Image: Flickr. ",
+      "A man is sitting in a bucket, while the ground is shaking, and some people are standing around him. He is",
+      "A skater and a girl, both wearing black, out on the street. $",
+      "A military officer is wearing a military uniform with a military badge. He is holding a gun with a satchel around",
+      "A large number of dogs and a small bird in a grassy field.",
+      "A strong female dog is near the hot red car.",
+      "A baseball player in a red uniform slides into the field as a baseball player in a green uniform approaches. Footage of",
+      "A young boy and a baseball player are swinging. \nAttachment: Tree. \nPhase: Summer. \nAge: ",
+      "A man wearing a clean shirt is cleaning the floor.",
+      "A tourist is riding in a boat on the beach with the sound of a smiling face on the screen of his or her",
+      "Three people are sitting on a mat with a tent in the background. \nAttachment: Beach. 5000",
+      "A man paints a portrait of a woman in a painted window pane. The room is decorated with a graffiti covered wall."
+    ],
+    "40": [
+      "20medjals of 2020s sized batches of pasta, making out 2020",
+      "A woman is holding her baby in the kitchen, and she is using the gun and the controls of the television. She",
+      "In a large field, a young man in a baseball uniform with a large field goal in his hand is surrounded by a",
+      "There is a large number of people at a table with a man eating a large dish. \n[+]",
+      "Three boys are wearing a dog's hat and a boat.",
+      "A child in a camera shoots at the camera while in the midst of a group of people.",
+      "某人正在用一条路旁的牌子。 交通指示器。 交通指示器。 交通指示器",
+      "The biker is on his bike and is doing a trick in the air while trying to make a funny pose. ",
+      "A man is in the office, he's holding his drink and talking on his cellphone. He's also in the bathroom",
+      "Girl and boy dancing on the dance floor with a red and white background. The light is coming from the left side of",
+      "The standing of a harness in the drawing, showing a person doing something.  Copyright 2006. ",
+      "A man with black, with a black kitchen, doing something, at a table. \nWashing machine, black,",
+      "A man with a red ball in a bowl holds a green ball in his hand.",
+      "A man looking at a sheet of plastic, as he examines a map of Europe. He looks at a picture of a",
+      "A person is sitting, while a large sand pile is scattered around, and a boat is standing still.",
+      "A guy, a girl, and a black guy, all on a street. They're all painting.",
+      "A military officer is wearing his military uniform and wearing a gold watch with his navy blue shirt. He is wearing a military",
+      "A large number of dogs in a grassy area. Several are jumping.  Captioned \"A Dog Fight\" Directed",
+      "The red dog is near the monster. red.",
+      "A baseball player in a blue uniform, a fielder in a white uniform, and a middle infielder in a gray",
+      "A young boy and a baseball player, both in a field.",
+      "A man wearing a clean shirt is cleaning the floor. \n\n//The man is sweeping the floor with a broom.",
+      "A person wearing a surfboard is standing on the beach with a rock in his hand while a group of people are walking",
+      "Three people (one with a yellow shirt) are spread-eagled on a mat, with one person's head on",
+      "A person is painting a room with a painted wall. The room is decorated with furniture. The person is using a paint"
+    ],
+    "60": [
+      "6 men, 20+ in 2009, 100 pounds each, 30",
+      "The woman is holding the gun and is trying to talk to her husband in the kitchen, and the man is in the",
+      "In a large field, a young man in a baseball uniform is in the air, while a large field is in the",
+      "There is a large number of people at a table with a man eating. 2 men are standing. 0",
+      "Three people are wearing a dog's coat.",
+      "A child in a bikini uses a camera in the face of a group of people while a third person watches.",
+      "A man is standing on a street with a yellow umbrella. \nA: yes\nB: no\nAnswer:\nA",
+      "The biker on the bike is doing a trick on the mountain and is making a big dip on the ground. Based",
+      "A man is in his office and is holding his phone and talking on the phone. He is also holding his hands in",
+      "A girl is dancing on the floor with a lighted stage and dark background. The girl is wearing a black dress.",
+      "The figure in the bottom of the frame being shown, standing, with a woman's head on top, and a mechanical",
+      "A guy, black, doing a kitchen, with a black kitchen, at a computer. \nSawing, doing",
+      "Three men hold a red ball in their hands. They are playing a game of tennis. 0",
+      "A man, in a field, looks at a map, reading a photo of a country. He looks at a picture",
+      "A person is sitting and standing behind a table. \n```\n```\n\nPlease determine whether the given text is related to",
+      "A guy, a black guy, and a guy doing something, at a black and white shop.",
+      "A man is wearing his military uniform and wearing a gold necklace. He is wearing the gold necklace on his shirt.",
+      "A large number of dogs in a field. Legend unknown.  Male dog in foreground.  Female dog behind.\nLarge",
+      "The redboned animal is red.",
+      "A baseball player in a field, a blond, and a man in a baseball uniform, in a field. The blond",
+      "一个青年和一只小鸟在一块儿。 带伞的男子。 青年和小鸟。 �",
+      "The man sweeping the floor with a clean shirt is cleaning the floor. \n//  The man is sweeping the floor with",
+      "A person wearing a surfboard is standing on the beach with a wave in the background while a camera is in the foreground",
+      "Three people, one with one hand on a sheet, one with one hand on a sheet, one with one hand on",
+      "The lady is painting the room and the walls are covered in paint."
+    ]
+  }
+}
\ No newline at end of file