protocol: freeze Oral-A v3 causal-capture funnel

6 files changed, 443 insertions, 0 deletions

diff --git a/ORAL_A_V3.md b/ORAL_A_V3.md
new file mode 100644
index 0000000..b668ea0
--- /dev/null
+++ b/ORAL_A_V3.md
@@ -0,0 +1,86 @@
+# Oral-A-v3: vectorizer-space causal calibration
+
+## Status and claim boundary
+
+Oral-A-v1 failed full ResNet training and Oral-A-v2 failed its frozen-forward
+early-layer causal-capture gate. Both outcomes remain failed. The post-v2
+oracle audit showed that learned channel-subspace calibration reached only
+`0.007209` early-third alignment, versus `0.023966` for the same vectorizer
+family fit directly to exact causal targets on a disjoint batch. V3 changes
+only this diagnosed regression interface: it estimates the A/G matrix target
+directly in the vectorizer parameter subspace rather than estimating
+per-example channel coefficients and regressing them afterward.
+
+V3 remains forward-only. Each direction uses the same two antithetic task-loss
+queries, K1/every-4 cadence, output-error context, channel-gated teaching field,
+local forward-weight eligibility, and no weight transport. No V3 GPU endpoint
+was observed before this protocol and its executable selectors were committed.
+The CIFAR-10 test set and seeds 10--14 remain untouched.
+
+## V3-0: exact mechanics and matched variance
+
+`experiments/conv_local_smoke.py` must verify before any endpoint:
+
+- the vectorizer-space estimator mean has cosine above `0.95` and norm ratio
+  in `[0.90,1.10]` to the exact A/G target;
+- at batch 128, its matched one-query MSE is below 25% of the
+  channel-coefficient-then-regress estimator's MSE;
+- no-BatchNorm and BatchNorm antithetic JVP relative errors are below `2e-6`;
+- implemented A/G updates agree with the analytic delta rule to absolute error
+  below `1e-14`;
+- every legacy convolutional/local-eligibility check remains green.
+
+This mechanics gate passed before V3-1 opened: cosine `0.969283`, norm ratio
+`1.048876`, MSE ratio `0.0338103`, JVP errors `1.61e-10` and `1.98e-10`, and
+delta-rule error `5.42e-20`.
+
+## V3-1: frozen-forward causal-capture gate
+
+Use the identical V2 initial-state protocol: seed-0 ResNet-20, first 10,000
+development-training examples, batch 128, zero-initialized channel-gated A/G,
+400 feedback-only minibatches, K1, `sigma=0.01`, perturbation seed 1000, and a
+fixed 64-example exact-gradient audit. Forward weights, readout, BN state, and
+BN affine parameters remain bitwise fixed.
+
+Run one matched V2 channel-subspace reference at `eta_A=0.01` and V3
+vectorizer-subspace candidates at `eta_A in {0.01,0.1,1.0}`. Select V3 by
+maximum early-third teaching/negative-gradient cosine, then maximum all-layer
+cosine, then lower rate. V3 advances only if all four records are finite and:
+
+1. selected V3 early-third alignment is at least `0.01`;
+2. selected V3 all-layer alignment is at least `0.05`;
+3. selected V3 early-third alignment exceeds the matched V2 reference by at
+   least `0.01`;
+4. selected V3 reaches at least 60% of the independently measured
+   channel-gated family oracle (`0.60 * 0.0239658 = 0.0143795`).
+
+The absolute-improvement condition is the binding anti-threshold-shopping
+rule: given V2's audited `0.007209`, V3 must reach approximately `0.017209`,
+not merely cross the old `0.01` threshold. No extra rate, warmup length,
+direction count, or feedback initialization is added after observation.
+
+## V3-2: full ResNet-20 validation gate
+
+Only after V3-1 passes, run one 200-epoch seed-0 model on all 45,000
+development-training examples. Copy the failed v1 schedule exactly: hidden LR
+`0.03`, output LR `0.1`, momentum `0.9`, weight decay `1e-4`, and 10x drops at
+epochs 100 and 150. Use selected V3 `eta_A`, 400 feedback-only warmup steps,
+and vectorizer-space K1/every-4 calibration. No recovery branch follows.
+
+Against the frozen BP (`91.62%`) and DFA (`33.06%`) records, V3 must have every
+metric finite, finish within 5 points of BP and at least 2 points above DFA,
+retain early-third alignment at least `0.05`, and use no more estimated MACs
+than BP. Failure closes V3 without test evaluation.
+
+## V3-3: untouched confirmation and score rule
+
+Only a V3-2 pass permits a new ResNet-20/32/56, seeds 10--14 confirmation with
+one final test evaluation per run and no depth-specific tuning. Its gate is
+identical to V2/V1: finite 45-run panel, SDIL within 2 points of BP at every
+depth, at least 2 points above DFA at depth 56, depth-20-to-56 degradation no
+worse than -2 points, retained early alignment, and a hardware-independent
+Pareto point.
+
+V3-0/V3-1 cannot raise the strict ICLR score. A complete V3-2 pass moves the
+forecast from 5 to 6; oral-level scale requires V3-3.
+
diff --git a/README.md b/README.md
index 61f42de..ad8e359 100644
--- a/README.md
+++ b/README.md
@@ -72,6 +72,7 @@ somatic statistic rather than arbitrary top-down context. `ROADMAP.md` and
 - `ROADMAP.md`: accept/oral evidence gates and their state;
 - `ORAL_A.md`: frozen standard CIFAR ResNet funnel;
 - `ORAL_A_V2.md`: frozen post-failure representable-subspace funnel;
+- `ORAL_A_V3.md`: frozen vectorizer-space causal-calibration funnel;
 - `REVIEW_SCORECARD.md`: adversarial ICLR-style score trajectory;
 - `results/figs/`: deterministic PDF/PNG main figures, captions, and a source
   hash manifest.
diff --git a/experiments/analyze_oral_a_v3_calibration.py b/experiments/analyze_oral_a_v3_calibration.py
new file mode 100644
index 0000000..c6b2fbf
--- /dev/null
+++ b/experiments/analyze_oral_a_v3_calibration.py
@@ -0,0 +1,134 @@
+#!/usr/bin/env python3
+"""Apply the frozen Oral-A-v3 causal-capture selector and gate."""
+import argparse
+import glob
+import json
+import math
+import os
+
+
+SPLIT_HASH = "8328b206a97c420e49e54e3eca4abe3274c4756b084355784ea3fb8059e4515b"
+ORACLE_EARLY = 0.02396580002561188
+
+
+def load(path):
+    with open(path) as handle:
+        record = json.load(handle)
+    args = record["args"]
+    expected = {
+        "mode": "sdil", "depth": 20, "width": 16, "seed": 0,
+        "epochs": 0, "train_limit": 10000, "val_examples": 5000,
+        "a_warmup_steps": 400, "pert_directions": 1, "pert_every": 4,
+        "pert_sigma": 0.01, "perturb_seed": 1000,
+        "normalization": "batchnorm", "vectorizer_mode": "channel_gated",
+        "a_scale": 0.0, "alignment_probe": 64,
+    }
+    for key, value in expected.items():
+        if args.get(key) != value:
+            raise ValueError(
+                f"{path}: {key}={args.get(key)!r}, expected {value!r}")
+    if record["provenance"]["git_tracked_dirty"]:
+        raise ValueError(f"tracked-dirty result: {path}")
+    if record["split"]["validation_index_sha256"] != SPLIT_HASH:
+        raise ValueError(f"split drift: {path}")
+    mode = args["apical_calibration_mode"]
+    expected_space = {
+        "channel_subspace": "channel_basis_moments",
+        "vectorizer_subspace": "vectorizer_parameter_gradients",
+    }[mode]
+    if record.get("calibration_metric_space") != expected_space:
+        raise ValueError(f"metric-space drift: {path}")
+    diagnostics = record.get("diagnostics")
+    warmup = record.get("apical_warmup", {}).get("mean")
+    if diagnostics is None or warmup is None:
+        raise ValueError(f"missing diagnostics/warmup aggregate: {path}")
+    values = diagnostics["teaching_negative_gradient_cosine"]
+    early_count = max(1, len(values) // 3)
+    metrics = {
+        "early_third_alignment": sum(values[:early_count]) / early_count,
+        "all_layer_alignment": sum(values) / len(values),
+        "mean_calibration_mse": warmup["calibration_mse"],
+        "mean_target_power": warmup["target_power"],
+        "mean_prediction_target_cosine": warmup["prediction_target_cosine"],
+        "mean_parameter_update_rms": warmup["parameter_update_rms"],
+    }
+    finite = (record["final"]["finite"]
+              and all(math.isfinite(value) for value in metrics.values()))
+    return {
+        "path": path, "source_commit": record["provenance"]["git_commit"],
+        "calibration_mode": mode, "eta_A": float(args["eta_A"]),
+        "metric_space": expected_space, "metrics": metrics, "finite": finite,
+    }
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--input", default="results/oral_a_v3_calibration")
+    parser.add_argument("--out", default="results/oral_a_v3_calibration_gate.json")
+    args = parser.parse_args()
+    rows = [load(path) for path in sorted(glob.glob(
+        os.path.join(args.input, "*.json")))]
+    observed = {(row["calibration_mode"], row["eta_A"]) for row in rows}
+    expected = {("channel_subspace", 0.01)} | {
+        ("vectorizer_subspace", rate) for rate in (0.01, 0.1, 1.0)}
+    if observed != expected or len(rows) != len(expected):
+        raise ValueError(
+            f"incomplete v3 grid: missing={expected-observed}, extra={observed-expected}")
+    if len({row["source_commit"] for row in rows}) != 1:
+        raise ValueError("v3 calibration source commits differ")
+    reference = next(row for row in rows
+                     if row["calibration_mode"] == "channel_subspace")
+    candidates = [row for row in rows
+                  if row["calibration_mode"] == "vectorizer_subspace"
+                  and row["finite"]]
+    candidates.sort(key=lambda row: (
+        -row["metrics"]["early_third_alignment"],
+        -row["metrics"]["all_layer_alignment"], row["eta_A"]))
+    selected = candidates[0] if candidates else None
+    checks = {
+        "all_four_records_finite": all(row["finite"] for row in rows),
+        "v3_candidate_selected": selected is not None,
+    }
+    if selected is not None:
+        metrics = selected["metrics"]
+        checks.update({
+            "v3_early_third_at_least_0.01": (
+                metrics["early_third_alignment"] >= 0.01),
+            "v3_all_layer_at_least_0.05": (
+                metrics["all_layer_alignment"] >= 0.05),
+            "v3_early_gain_over_v2_at_least_0.01": (
+                metrics["early_third_alignment"]
+                - reference["metrics"]["early_third_alignment"] >= 0.01),
+            "v3_reaches_60pct_of_family_oracle": (
+                metrics["early_third_alignment"] >= 0.60 * ORACLE_EARLY),
+        })
+    else:
+        checks.update({
+            "v3_early_third_at_least_0.01": False,
+            "v3_all_layer_at_least_0.05": False,
+            "v3_early_gain_over_v2_at_least_0.01": False,
+            "v3_reaches_60pct_of_family_oracle": False,
+        })
+    passed = all(checks.values())
+    output = {
+        "protocol": "oral_a_v3_vectorizer_causal_capture_v1",
+        "status": "passed" if passed else "failed",
+        "checks": checks, "rows": rows, "matched_v2_reference": reference,
+        "selected_v3": selected, "family_oracle_early_third": ORACLE_EARLY,
+        "confirmation_test_seeds_touched": False,
+        "review_score_before": 5, "review_score_after": 5,
+        "score_change_rule": "mechanics/calibration alone cannot raise score",
+    }
+    os.makedirs(os.path.dirname(os.path.abspath(args.out)), exist_ok=True)
+    with open(args.out, "w") as handle:
+        json.dump(output, handle, indent=2, sort_keys=True)
+        handle.write("\n")
+    print(json.dumps({
+        "status": output["status"], "checks": checks,
+        "reference": reference, "selected_v3": selected,
+    }, indent=2))
+
+
+if __name__ == "__main__":
+    main()
+
diff --git a/experiments/analyze_oral_a_v3_full.py b/experiments/analyze_oral_a_v3_full.py
new file mode 100644
index 0000000..f24e5d9
--- /dev/null
+++ b/experiments/analyze_oral_a_v3_full.py
@@ -0,0 +1,113 @@
+#!/usr/bin/env python3
+"""Apply the frozen Oral-A-v3 full-validation gate."""
+import argparse
+import json
+import math
+import os
+
+
+SPLIT_HASH = "8328b206a97c420e49e54e3eca4abe3274c4756b084355784ea3fb8059e4515b"
+
+
+def finite_tree(value):
+    if isinstance(value, dict):
+        return all(finite_tree(item) for item in value.values())
+    if isinstance(value, list):
+        return all(finite_tree(item) for item in value)
+    if isinstance(value, float):
+        return math.isfinite(value)
+    return True
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument(
+        "--run", default="results/oral_a_v3_dev/sdil_full_r20_s0.json")
+    parser.add_argument(
+        "--selection", default="results/oral_a_v3_calibration_gate.json")
+    parser.add_argument(
+        "--bp", default="results/oral_a_dev/bp_reference_primary.json")
+    parser.add_argument(
+        "--dfa", default="results/oral_a_dev/dfa_full_r20_s0.json")
+    parser.add_argument("--out", default="results/oral_a_v3_full_gate.json")
+    args = parser.parse_args()
+    with open(args.run) as handle:
+        run = json.load(handle)
+    with open(args.selection) as handle:
+        selection = json.load(handle)
+    with open(args.bp) as handle:
+        bp = json.load(handle)
+    with open(args.dfa) as handle:
+        dfa = json.load(handle)
+    if selection["status"] != "passed":
+        raise ValueError("v3 causal-capture gate did not pass")
+    chosen = selection["selected_v3"]
+    expected = {
+        "mode": "sdil", "depth": 20, "width": 16, "seed": 0,
+        "epochs": 200, "val_examples": 5000, "lr": 0.03,
+        "output_lr": 0.1, "lr_schedule": "step",
+        "lr_milestones": "100,150", "lr_gamma": 0.1,
+        "a_scale": 0.0, "eta_A": chosen["eta_A"],
+        "a_warmup_steps": 400,
+        "apical_calibration_mode": "vectorizer_subspace",
+        "pert_sigma": 0.01, "pert_directions": 1, "pert_every": 4,
+        "normalization": "batchnorm", "vectorizer_mode": "channel_gated",
+    }
+    for key, value in expected.items():
+        if run["args"].get(key) != value:
+            raise ValueError(
+                f"v3 run {key}={run['args'].get(key)!r}, expected {value!r}")
+    if run["provenance"]["git_tracked_dirty"]:
+        raise ValueError("tracked-dirty v3 result")
+    if run["split"]["validation_index_sha256"] != SPLIT_HASH:
+        raise ValueError("v3 split drift")
+    if run["evaluation_protocol"]["test_evaluations"] != 0:
+        raise ValueError("test endpoint touched during v3 development")
+    values = run["diagnostics"]["teaching_negative_gradient_cosine"]
+    early_count = max(1, len(values) // 3)
+    early = sum(values[:early_count]) / early_count
+    sdil_accuracy = run["final"]["accuracy"]
+    bp_accuracy = bp["final"]["accuracy"]
+    dfa_accuracy = dfa["final"]["accuracy"]
+    checks = {
+        "all_metrics_finite": run["final"]["finite"] and finite_tree(run),
+        "bp_reference_at_least_90pct": bp_accuracy >= 0.90,
+        "sdil_within_5pt_of_bp": sdil_accuracy >= bp_accuracy - 0.05,
+        "sdil_at_least_2pt_above_dfa": sdil_accuracy >= dfa_accuracy + 0.02,
+        "early_third_alignment_at_least_0.05": early >= 0.05,
+        "sdil_macs_no_more_than_bp": (
+            run["work"]["total_macs_estimate"]
+            <= bp["work"]["total_macs_estimate"]),
+    }
+    passed = all(checks.values())
+    output = {
+        "protocol": "oral_a_v3_full_validation_v1",
+        "status": "passed" if passed else "failed", "checks": checks,
+        "metrics": {
+            "sdil_validation_accuracy": sdil_accuracy,
+            "bp_validation_accuracy": bp_accuracy,
+            "dfa_validation_accuracy": dfa_accuracy,
+            "early_third_alignment": early,
+            "sdil_total_macs": run["work"]["total_macs_estimate"],
+            "bp_total_macs": bp["work"]["total_macs_estimate"],
+        },
+        "sources": {"sdil": args.run, "bp": args.bp, "dfa": args.dfa},
+        "confirmation_test_seeds_touched": False,
+        "review_score_before": 5, "review_score_after": 6 if passed else 5,
+        "review_score_rationale": (
+            "full standard-scale validation passed; independent depth panel remains"
+            if passed else "full standard-scale validation failed"),
+    }
+    os.makedirs(os.path.dirname(os.path.abspath(args.out)), exist_ok=True)
+    with open(args.out, "w") as handle:
+        json.dump(output, handle, indent=2, sort_keys=True)
+        handle.write("\n")
+    print(json.dumps({
+        "status": output["status"], "checks": checks,
+        "metrics": output["metrics"],
+    }, indent=2))
+
+
+if __name__ == "__main__":
+    main()
+
diff --git a/experiments/oral_a_v3_calibration_screen.py b/experiments/oral_a_v3_calibration_screen.py
new file mode 100644
index 0000000..d353b3c
--- /dev/null
+++ b/experiments/oral_a_v3_calibration_screen.py
@@ -0,0 +1,60 @@
+#!/usr/bin/env python3
+"""Run a shard of the frozen Oral-A-v3 causal-capture screen."""
+import argparse
+import os
+import subprocess
+import sys
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--device", default="cuda")
+    parser.add_argument("--shard_index", type=int, default=0)
+    parser.add_argument("--num_shards", type=int, default=1)
+    parser.add_argument("--dry_run", action="store_true")
+    args = parser.parse_args()
+    if not 0 <= args.shard_index < args.num_shards:
+        raise ValueError("invalid shard index")
+    common = [
+        sys.executable, "experiments/conv_run.py",
+        "--mode", "sdil", "--device", args.device,
+        "--depth", "20", "--width", "16", "--seed", "0",
+        "--loader_seed", "0", "--batch_size", "128", "--epochs", "0",
+        "--train_limit", "10000", "--val_examples", "5000",
+        "--split_seed", "2027", "--eval_split", "validation",
+        "--eval_every", "0", "--augment_train", "1",
+        "--lr", "0.03", "--output_lr", "0.1",
+        "--lr_schedule", "constant", "--warmup_epochs", "0",
+        "--momentum", "0.9", "--weight_decay", "1e-4",
+        "--normalization", "batchnorm", "--vectorizer_mode", "channel_gated",
+        "--a_scale", "0", "--a_warmup_steps", "400",
+        "--pert_sigma", "0.01", "--pert_directions", "1",
+        "--pert_every", "4", "--perturb_seed", "1000",
+        "--alignment_probe", "64",
+    ]
+    jobs = [
+        ("channel_subspace_etaA0.01", common + [
+            "--apical_calibration_mode", "channel_subspace",
+            "--eta_A", "0.01",
+            "--out", "results/oral_a_v3_calibration/channel_subspace_etaA0.01.json",
+        ])
+    ]
+    for rate in (0.01, 0.1, 1.0):
+        tag = f"vectorizer_subspace_etaA{rate}"
+        jobs.append((tag, common + [
+            "--apical_calibration_mode", "vectorizer_subspace",
+            "--eta_A", str(rate),
+            "--out", f"results/oral_a_v3_calibration/{tag}.json",
+        ]))
+    os.makedirs("results/oral_a_v3_calibration", exist_ok=True)
+    for index, (tag, command) in enumerate(jobs):
+        if index % args.num_shards != args.shard_index:
+            continue
+        print(tag, " ".join(command), flush=True)
+        if not args.dry_run:
+            subprocess.run(command, check=True)
+
+
+if __name__ == "__main__":
+    main()
+
diff --git a/experiments/oral_a_v3_full_development.py b/experiments/oral_a_v3_full_development.py
new file mode 100644
index 0000000..0b38f7d
--- /dev/null
+++ b/experiments/oral_a_v3_full_development.py
@@ -0,0 +1,49 @@
+#!/usr/bin/env python3
+"""Run the single frozen Oral-A-v3 full ResNet-20 validation job."""
+import argparse
+import json
+import os
+import subprocess
+import sys
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument(
+        "--selection", default="results/oral_a_v3_calibration_gate.json")
+    parser.add_argument("--device", default="cuda")
+    parser.add_argument("--dry_run", action="store_true")
+    args = parser.parse_args()
+    with open(args.selection) as handle:
+        selection = json.load(handle)
+    if selection["status"] != "passed":
+        raise ValueError("Oral-A-v3 causal-capture gate did not pass")
+    chosen = selection["selected_v3"]
+    command = [
+        sys.executable, "experiments/conv_run.py",
+        "--mode", "sdil", "--device", args.device,
+        "--depth", "20", "--width", "16", "--seed", "0",
+        "--loader_seed", "0", "--batch_size", "128", "--epochs", "200",
+        "--val_examples", "5000", "--split_seed", "2027",
+        "--eval_split", "validation", "--eval_every", "20",
+        "--augment_train", "1", "--lr", "0.03", "--output_lr", "0.1",
+        "--lr_schedule", "step", "--lr_milestones", "100,150",
+        "--lr_gamma", "0.1", "--warmup_epochs", "0", "--momentum", "0.9",
+        "--weight_decay", "1e-4", "--normalization", "batchnorm",
+        "--vectorizer_mode", "channel_gated", "--a_scale", "0",
+        "--eta_A", str(chosen["eta_A"]), "--a_warmup_steps", "400",
+        "--apical_calibration_mode", "vectorizer_subspace",
+        "--pert_sigma", "0.01", "--pert_directions", "1",
+        "--pert_every", "4", "--perturb_seed", "1000",
+        "--alignment_probe", "32",
+        "--out", "results/oral_a_v3_dev/sdil_full_r20_s0.json",
+    ]
+    os.makedirs("results/oral_a_v3_dev", exist_ok=True)
+    print(" ".join(command), flush=True)
+    if not args.dry_run:
+        subprocess.run(command, check=True)
+
+
+if __name__ == "__main__":
+    main()
+