World Alignment: unpaired cross-modal correspondence by relational identifiability

Method: scene states are sets of part states; relation fields are built
within each modality and are invariant to how each side labels its own
features; the cross-modal bridge is a coupling searched under an energy
that is a closed-form functional of one matrix; solving is spectral
initialisation followed by exact local refinement.

Evidence: in a procedurally generated closed world, blind recovery of a
hidden image-caption correspondence reaches 95.3% at 256 scenes against
0.39% chance, and the recovered pairs transfer to 200 held-out scenes at
93.0% exact retrieval with random-pair and shuffled-image controls at or
near chance. Cross-modal value correspondence is derived from disjoint
corpora rather than declared. On Visual Genome the field correlation
reaches 0.656 against the 0.9 that polynomial recovery needs, with the
deficit attributed away from segmentation and discretisation.

Protocol: no image-text pair enters any objective, optimiser,
initialisation, or model selection; hidden pairs score orderings only.

Co-Authored-By: Claude <noreply@anthropic.com>

1 files changed, 337 insertions, 0 deletions

diff --git a/artifacts/energy_orbit_prefix_val_m30.json b/artifacts/energy_orbit_prefix_val_m30.json
new file mode 100644
index 0000000..d1a1ee3
--- /dev/null
+++ b/artifacts/energy_orbit_prefix_val_m30.json
@@ -0,0 +1,337 @@
+{
+  "samples": 100,
+  "mean_best_reference_unigram_f1": {
+    "initial": 0.24513073565703838,
+    "final": 0.2559508105017754,
+    "oracle": 0.5184897529033539
+  },
+  "energy_protocol": "No cross-modal map and no image-text pair is used by the energy or optimizer. Paired text is loaded only for trajectory and oracle diagnostics; the final step is fixed by CLI arguments.",
+  "prefix_training": "text only; no image features or image-text pairs",
+  "examples": [
+    {
+      "row": 67,
+      "references": [
+        "A group of people stand in the back of a truck filled with cotton.",
+        "Men are standing on and about a truck carrying a white substance.",
+        "A group of people are standing on a pile of wool in a truck.",
+        "A group of men are loading cotton onto a truck",
+        "Workers load sheared wool onto a truck."
+      ],
+      "initial": "Two women make and use small breads.  They are making a necklace.  They are both young. \n A: yes B: no\nYou are",
+      "final": "A boy wears a black shirt and sits on a fence in front of a market in Italy. The boy is looking at a red and white pig. In the",
+      "oracle": "A group of people are loading a truck with cotton on top of a pile of steel."
+    },
+    {
+      "row": 99,
+      "references": [
+        "An adult wearing a gray shirt with red sleeves sleeping on a couch.",
+        "A woman in black pants napping on the couch.",
+        "A man sleeping in a green room on a couch.",
+        "The young woman is sleeping in her room.",
+        "A long-haired man sleeping on a couch"
+      ],
+      "initial": "A young boy is playing with a green and yellow pipe in a suburban area.",
+      "final": "This Asian man is cutting a piece of meat with a knife that is being thrown by another man.  He is obviously very experienced in this kind of work.",
+      "oracle": "A sleepy woman in a yellow shirt sleeping on a bed."
+    },
+    {
+      "row": 100,
+      "references": [
+        "Boy in brown shirt with headphones on sits on woman's shoulders in a crowd.",
+        "A kid wearing headphones sits on his mom's shoulders while in a crowd.",
+        "A boy with headphones on sitting on top of a woman's shoulders.",
+        "A child with headphones riding on his mother's shoulders.",
+        "A boy wearing headphones sits on a woman's shoulders."
+      ],
+      "initial": "Two people in a conference room microphone speak to a large audience.  Source: YouTube. 00:00.00.00.0",
+      "final": "A person jumps off a skateboard on a blue wall with a nearby fountain. \nBrainMass Answers\n\n## Questions related to the topic 3D printing\n\n1",
+      "oracle": "A boy with headphones on sitting on a girl's head while riding a bike."
+    },
+    {
+      "row": 109,
+      "references": [
+        "Two men, standing on an ice, looking into something covered with a blue tarp.",
+        "Two men are about to enter an ice fishing tent on a snow covered lake.",
+        "Two people standing outside a blue tent structure on a snowy surface.",
+        "Two men setting up a blue ice fishing hut on an iced over lake",
+        "A tent is being set up on the ice."
+      ],
+      "initial": "A group of men are sitting around a campfire eating food. They all appear to be wearing blue jackets.",
+      "final": "Two women wearing red dresses hold their mouths while running behind a red bike in the field. Caption: Red Dressed Runners Running Behind a Red Bike. Free",
+      "oracle": "Two men are laying on a tent on a frozen lake that is being heated up. \nOptions: (A) on the ground (B) on the ice"
+    },
+    {
+      "row": 118,
+      "references": [
+        "A man in a green shirt and red life jacket is sitting in a canoe drifting around the lake.",
+        "A man wearing a life jacket is in a small boat on a lake with a ferry in view.",
+        "A man wearing a red life-vest is sitting in a canoe on a body of water.",
+        "A balding man wearing a red life jacket is sitting in a small boat.",
+        "A man in a red life vest is riding in a canoe."
+      ],
+      "initial": "Two people looking over the crowd of people holding a glass of beer.",
+      "final": "Two women dressed in red and blue wait for something on the wall. Two women dressed in red and blue wait for something on the wall. 5 1",
+      "oracle": "A man in a blue life jacket is sitting in a raft on a river."
+    },
+    {
+      "row": 161,
+      "references": [
+        "A woman in jeans and a red coat and carrying a multicolored handbag spreads put her arms while leaping up from a cobblestone street.",
+        "A lady in a red coat, holding a bluish hand bag likely of asian descent, jumping off the ground for a snapshot.",
+        "A woman in a red jacket, with a blue and gold handbag is jumping while smiling on a cobblestone street.",
+        "A woman wearing a red coat is jumping outdoors.",
+        "Girl in red jumping for joy."
+      ],
+      "initial": "A brown dog runs in the forest with his leash. Laborious in the background. License: CC 3.0.0.0.0. License",
+      "final": "A man in a black shirt with a yellow backpack doing a trick with his feet. 500 meters ahead. \nOptions:\n- yes\n- no",
+      "oracle": "A woman in a red jacket and blue jeans jumping off a bridge."
+    },
+    {
+      "row": 170,
+      "references": [
+        "Two dogs run towards each other on a rocky area with water in the background.",
+        "A brown dog is running after a black dog on a rocky shore.",
+        "Two dogs run across stones near a body of water.",
+        "A brown dog is running after the black dog.",
+        "Two dogs playing on a beach."
+      ],
+      "initial": "Two women wearing red blankets and sandals are sitting on a wooden structure. They are playing with a fire. \nClassification: Natural. \nClassification: Natural. \n",
+      "final": "A girl in a pink and white shirt is wearing a pink bow and a white headband with a pink bow on it. She is holding a pink and white",
+      "oracle": "Two dogs are running on a rocky beach next to a water source. Male and female dogs are running together. Scene is in a wooded area."
+    },
+    {
+      "row": 200,
+      "references": [
+        "A young helmeted man, in his team uniform, is swinging his bat at an incoming baseball.",
+        "A young boy wearing a Giants jersey swings a baseball bat at an incoming pitch.",
+        "A boy playing for the Giants baseball team swinging at a baseball.",
+        "A child swings a baseball bat at the ball.",
+        "This little boy hitting a ball."
+      ],
+      "initial": "A bike rider doing a trick on a bicycle in the background. Motivated to be cool. \n\n\nOutput:\n![bike rider doing trick](https://i.stack",
+      "final": "A boy with a red shirt and blue shoes is playing in the water of the playground.",
+      "oracle": "A young boy swings a baseball at a ball hit by a ball player. The boy is wearing a striped shirt."
+    },
+    {
+      "row": 203,
+      "references": [
+        "A guy sitting at a desk talking on a phone with books laying everywhere.",
+        "Man talking on a phone, surrounded by books in an office.",
+        "A man sits at a desk in room cluttered with books.",
+        "A man on the phone surrounded by stacks of books.",
+        "A man in a cluttered office is using the telephone"
+      ],
+      "initial": "A brown and white man with a camera and a gray shirt is walking around. He is holding a cellphone. \nAnswer: Step 1: Identify the main",
+      "final": "An Asian woman is performing a traditional martial arts exercise with a group of young men. \nCategory: Foreigners  A: Yes B: No Question: Is",
+      "oracle": "A man in a chair at a desk talking on his cellphone. He is standing in a cluttered room.} \n# A man in a chair talking on"
+    },
+    {
+      "row": 257,
+      "references": [
+        "Woman is enjoying herself being outdoors biking in a forrest preserve.",
+        "A girl in an orange tank top is walking her bike through the forrest.",
+        "A smiling woman in a peach tank top stands holding a mountain bike",
+        "A happy teenage girl taking a break from her bike ride.",
+        "A girl smiles while holding her bike in a field."
+      ],
+      "initial": "A couple in a pink car, with a white and black umbrella on the side.  Neighbors watching.  Pedestrian.  Pink car. ",
+      "final": "Person in white hat and shorts with a striped shirt stands with friends. Image is blurry. \n\n##### Suggestion: \nStep 1: Identify the subject of",
+      "oracle": "A girl in a pink shirt is smiling while riding a bike in a park. Motto: Happy to be a cyclist.  Motto: Happy to be"
+    },
+    {
+      "row": 280,
+      "references": [
+        "A boy wearing blue and yellow walking on a cliff edge.",
+        "A boy in yellow shorts is standing on top of a cliff.",
+        "A young child is standing alone on some jagged rocks.",
+        "A little boy standing high in the air on a rock.",
+        "Child stands near edge of cliff."
+      ],
+      "initial": "A man sits on a large blanket near a statue of a man with a large hand. \n##### Related Questions:\n\n1. ### Math\n\nA man is standing",
+      "final": "A blue-haired, middle-aged man stands in a narrow, stone-lined courtyard near a stone wall. The man is leaning against the wall. The scene is in",
+      "oracle": "A young boy standing on a cliff ledge with blue shorts."
+    },
+    {
+      "row": 282,
+      "references": [
+        "A snowmobile rider flies through the air on his or her machine in front of tall pine trees.",
+        "Rider jumps snowmobile high in rural area.",
+        "A person on a snowmobile in mid jump.",
+        "A snowmobiler flies through the air.",
+        "The man is up in the air."
+      ],
+      "initial": "A couple with a weird-looking car, a man with a big, orange hat, and a woman with a weird-looking face, all in a window. They",
+      "final": "A man in a black suit and a woman in a black dress hold a small boy in a black jacket and a black mask in a parking lot.",
+      "oracle": "A snowboarder flies high in the air on a mountain. \n;; \nmountain biking near the snow\nExplanation: The given text is a description of"
+    },
+    {
+      "row": 293,
+      "references": [
+        "A mother and three children collecting garbage from a blue and white garbage can on the street.",
+        "A woman and three children are picking something out of a blue trashcan.",
+        "Three young children stand around a blue and white barrel.",
+        "Four children are standing around a blue and white barrel.",
+        "A group of children look into a barrel."
+      ],
+      "initial": "A blond boy in a blue shirt is in a white box while being introduced to a large white box.",
+      "final": "A man sitting on a red and blue cushion is holding a red and white umbrella.",
+      "oracle": "Three children are looking up at a blue and white container."
+    },
+    {
+      "row": 301,
+      "references": [
+        "A dark-haired woman in a green and white apron sits amid her outdoor display of flowers for sale on a city street.",
+        "A woman is sitting by her dried flower display at an outside market.",
+        "A woman sells flowers and incense at the market.",
+        "A lady is sitting down tending to her stand.",
+        "A woman in an apron shopping at a market."
+      ],
+      "initial": "A man lays blue and gray on the sand to build a pile of sand. He uses a shovel.  A blue and gray man lays on the sand",
+      "final": "A man with a black shirt and a blue helmet checks out a car with his cellphone. The car is a 2010 Ford F-15",
+      "oracle": "A woman is sitting at a flower stand in the market. She is working on her garden."
+    },
+    {
+      "row": 322,
+      "references": [
+        "There is a woman and man wearing orange shirts, playing instruments with sheet paper in front of them.",
+        "A cello player is playing during a concert.",
+        "A woman in a red shirt playing the cello.",
+        "A female playing a song on her violin.",
+        "A woman in a red shirt plays a cello."
+      ],
+      "initial": "A player in a red jacket playing with a pipe while other people watch. \nSentence: A man in a red jacket is playing with a pipe while other people",
+      "final": "Three individuals are standing and walking in an outdoor setting, with an ensemble of instruments being played. \nThere is also a small group of people sitting in the background",
+      "oracle": "A woman playing the cello in a black shirt."
+    },
+    {
+      "row": 370,
+      "references": [
+        "People on ATVs and dirt bikes are traveling along a worn path in a field surrounded by trees.",
+        "Three people on two dirt-bikes and one four-wheeler are riding through brown grass.",
+        "Three people ride off-road bikes through a field surrounded by trees.",
+        "Three people on motorbikes follow a trail through dry grass.",
+        "Three people are riding around on ATV's and motorcycles."
+      ],
+      "initial": "Two young men in t-shirts ride on ice in front of a crowd of people. Title: Ice Skating in the Snow\n\nCaption: Two young men in",
+      "final": "A man in a red shirt is holding a camera next to a couple walking on a sidewalk in front of a brick building.",
+      "oracle": "Three people are riding on dirt bikes around a dirt road. They are wearing helmets. \nChoices: (a). Yes (b). No;\nWould the answer"
+    },
+    {
+      "row": 377,
+      "references": [
+        "An older man sits back and relaxes on a patio outside an adobe building where many bicycles are propped.",
+        "Topless old man wearing slippers, navy blue pants and a white hat while reclining in a chair outside.",
+        "A shirtless man in a white cap relaxes in a deck chair, close to three parked bicycles.",
+        "A half naked man is sleeping on his chair outdoors.",
+        "Man relaxing in a folding chair on the street."
+      ],
+      "initial": "The man with a white beard and military-style dress has a white headband and a military-style vest. He is wearing a light jacket and a light hat.",
+      "final": "A man wearing a blue shirt and hat poses for a bike ride. \nFilename: 1001001.jpg\nQuestion: What does the",
+      "oracle": "A relaxed man in shorts lounges in a pool of water while lying down on a towel. \n####\nA relaxed man lounges in a pool of water while"
+    },
+    {
+      "row": 382,
+      "references": [
+        "A group of people are standing outside at a carnival, near a ride called the Green Jungle.",
+        "A crowd of people standing next to a shack labeled \"Green Jungle.\"",
+        "A group of people standing in front of a hut in a parking lot.",
+        "Crowd standing near \"GREEN JUNGLE\" exhibit at festival.",
+        "A group of people are gathered."
+      ],
+      "initial": "A large dog is lying in a field next to a small brown dog. Large dog is next to a small brown dog. \nQuestion: What is the relationship",
+      "final": "A girl rides a skateboard on a tireless ramp in front of a green field. Caption: Skateboarder on a tireless ramp.",
+      "oracle": "A group of people standing in a crowd of people near a sign."
+    },
+    {
+      "row": 387,
+      "references": [
+        "A young girl sitting on a small chair surrounded by what looks like handmade fabrics and bags.",
+        "A girl wearing a colorful hat sits on a woven mat.",
+        "A young woman is making rugs in the rain forest",
+        "A woman in a red hat sits on a rug.",
+        "A woman sitting in a hut."
+      ],
+      "initial": "A black dog is cowed by a brown and white dog.",
+      "final": "A man in a white shirt is standing in front of a large cardboard box. He is holding a folder. \n## Step 3: Create a Question for",
+      "oracle": "A woman in a pink scarf sits on a wooden floor. \nA woman sitting on a wooden floor wearing a pink scarf."
+    },
+    {
+      "row": 390,
+      "references": [
+        "There are three young people, one dark-haired holding a bottled drink, one with head tilted back and finger in her mouth, and one with spiked hair.",
+        "Three girls make faces as one takes a drink while they stand in a busy street.",
+        "Three young women pose for the camera at a parade.",
+        "Three women on the street drinking and posing",
+        "The young people party in the streets."
+      ],
+      "initial": "A baseball player swings to make contact with the catcher as he is out of bounds. (USA Baseball) Category: Basketball, Baseball, Baseball Player. Description:",
+      "final": "A person is falling off a skateboard on a small hill with a small pool in the background.",
+      "oracle": "Three young women stand in the crowd and take a picture. They are all smiling."
+    },
+    {
+      "row": 419,
+      "references": [
+        "A group of people gather in a bar, and one man in a black shirt holds his arm and signals to the camera.",
+        "In a crowd of young people at a bar, a man stands displaying the rock and roll sign.",
+        "A man is making a \"rock\" gesture while standing on a stool in a crowded bar.",
+        "A single man in a black t-shirt standing above the crowd at a busy bar.",
+        "People enjoying drinks in a crowded bar."
+      ],
+      "initial": "A little boy in a brown shirt is playing with a chain saw. He is jumping.",
+      "final": "A woman is submerged in water, her body is partially submerged in a pool, and a person is standing on the side of the pool, looking down at the",
+      "oracle": "A group of people in a bar drinking in a crowd, one stands on a table."
+    },
+    {
+      "row": 422,
+      "references": [
+        "Woman and man walking across wooden rope bridge with a caution sign beside it.",
+        "A man and a woman are crossing over a rope bridge with greenery all over them.",
+        "A man and a woman crossing a suspension bridge in a tropical setting.",
+        "A bridge through high green plants, a man and a woman on it.",
+        "A man and a woman are walking across a rope bridge."
+      ],
+      "initial": "A couple is cleaning on the lower tubs, with a blue shirt. Friendly. \"",
+      "final": "A_man_in_a_lawn_playing_a_guitar_on_his_boat_sitting_on_a bench.",
+      "oracle": "A man and a woman cross a bridge over a river and walk under a bridge. \nAttachment:"
+    },
+    {
+      "row": 462,
+      "references": [
+        "A man in a gray cap, a pink tank top and short skirt, pink-and-black striped socks, and boots juggles knives in front of a brick building whose sign reads \"The Pump Room\" while onlookers watch.",
+        "An male entertainer is wearing women's clothing that consists of a pink tank top, skirt, knee high socks, and a hairnet, performing on a unicycle a juggling knives while a crowd watches.",
+        "A man, wearing a short pink skirt, long striped socks, a pink tank top, and a hairnet, is juggling knives to the amusement of onlookers.",
+        "A street juggler performing while wearing a colorful outfit.",
+        "A man in a skirt is jumping while juggling knives."
+      ],
+      "initial": "A girl with a pink shirt is sitting in a shopping cart on a table in a store.",
+      "final": "A boy running a race with his feet out in the air and a black dog doing the same.",
+      "oracle": "A man wearing a striped shirt, shorts, and a hat is performing tricks on a trampoline."
+    },
+    {
+      "row": 478,
+      "references": [
+        "Woman dressed in a green hat and apron is holding a tray full of drink samples.",
+        "An Asian girl in a green hat and apron is serving drinks on a tray.",
+        "The waitress offers complimentary tea to the patrons.",
+        "A girl hands out Starbucks samples at a street fair.",
+        "A Starbucks barista offering up a tray of drinks"
+      ],
+      "initial": "Three people are wearing sandals and are setting up a lawn chair. The setting is in the middle of the grass. The weather is sunny. The photographer is behind",
+      "final": "Two men sit at a table in a room, one is holding a computer and the other is looking at a piece of paper in front of him. The table",
+      "oracle": "A waitress in a white shirt and apron serves coffee to people at a buffet table. The waitress is holding a tray."
+    },
+    {
+      "row": 511,
+      "references": [
+        "Construction workers standing on top of a piece of machinery.",
+        "Two Rubbish men standing on theyre truck loading trash.",
+        "A blurry picture of a man in a blue shirt and hard hat.",
+        "A man in a white hat stands on top of a truck.",
+        "Two people are on top of the bed of a truck."
+      ],
+      "initial": "A man, in a green jacket, works on a large, square area of black and white tiles. The man is about 6 feet tall.  Description",
+      "final": "A crowd of people dressed in festive attire wait on a street lined with buildings and trees during a rainy day. Captioned by a street artist. 20",
+      "oracle": "A man in a shirt and hat standing on top of a truck."
+    }
+  ]
+}
\ No newline at end of file