summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorYurenHao0426 <Blackhao0426@gmail.com>2026-07-22 20:32:19 -0500
committerYurenHao0426 <Blackhao0426@gmail.com>2026-07-22 20:32:19 -0500
commit32122d026176ed199271aea243ccb0eef78eb30f (patch)
treeaaa78f5439625d7f39663a38275eacc447d85eb4
parent9ca51a97119efc90bc2a0b6b65624dc87689e088 (diff)
protocol: freeze oral B recovery confirmation
-rw-r--r--ORAL_B_RECOVERY.md58
-rw-r--r--ROADMAP.md7
-rwxr-xr-xexperiments/analyze_bci_td_confirmation.py374
-rwxr-xr-xexperiments/bci_td_confirmation.py199
-rwxr-xr-xexperiments/bci_td_confirmation.sh16
-rwxr-xr-xexperiments/finalize_accept.sh4
6 files changed, 657 insertions, 1 deletions
diff --git a/ORAL_B_RECOVERY.md b/ORAL_B_RECOVERY.md
index 7ab6785..da018dd 100644
--- a/ORAL_B_RECOVERY.md
+++ b/ORAL_B_RECOVERY.md
@@ -118,3 +118,61 @@ necessity, innovation identification, and a causal plasticity lesion.
The executable runner, complete-grid analyzer, and shell entry point are
`experiments/bci_td_run.py`, `experiments/analyze_bci_td_development.py`, and
`experiments/bci_td_development_screen.sh`. None may run before D4 closes.
+
+## R2 frozen untouched confirmation
+
+R2 is committed before any R1 task endpoint and is executable only if the
+complete R1 gate selects one eligible rate. It runs that rate without further
+selection over untouched task seeds 10--15 and model seeds 0--4: exactly 30
+paired records, with no deletion or replacement. Each record repeats intact,
+fixed-vectorizer, plasticity-lesion, and exact-role diagnostic conditions on
+identical trajectories. Its separate 256-episode evaluation uses task seed
+`300000 + training_task_seed`. The R2 runner requires clean tracked code,
+protocol, D4 gate, and R1 gate and binds all of them by SHA-256.
+
+The task seed, rather than each of the five models sharing its environment, is
+the independent unit for uncertainty. Every aggregate confidence check first
+averages model seeds within each of the six task seeds and then uses a
+one-sided 95% Student-t bound with five degrees of freedom. R2 requires all of
+the following learning/plasticity checks:
+
+- mean intact final success at least 70%, every task mean at least 60%, and
+ the lower confidence bound at least 60%;
+- mean early-to-late learning gain at least 10 points and its lower bound at
+ least 5 points;
+- mean paired final gap over fixed vectorizer at least 20 points and its lower
+ bound at least 10 points;
+- mean deficit to the exact-role diagnostic at most 10 points and its upper
+ bound at most 20 points;
+- the lower bound of `0.5 * intact_gain - plasticity_lesion_gain` is
+ nonnegative; and
+- mean learned-role cosine at least 0.80 and every one of 30 cosines at least
+ 0.70.
+
+The biological checks retain the thresholds frozen for the original B1/B2
+programme and add only task-cluster robustness bounds:
+
+- mean absolute innovation-soma correlation at most 0.10 (upper bound 0.12),
+ with mean raw-minus-innovation correlation at least 0.20 (lower bound 0.15);
+- preceding surrounding-network state predicts amplification at mean balanced
+ accuracy at least 55% (lower bound 52%), while decoder distance has mean
+ residual correlation at least 0.10 (lower bound 0.02);
+- residual population outcome accuracy averages at least 57% (lower bound
+ 53%) and exceeds soma by at least 3 points on average with nonnegative lower
+ bound;
+- causal-role sign inversion is positive in at least 25/30 records and in all
+ six task-cluster means;
+- performance velocity has at least 0.05 mean absolute CV-correlation
+ advantage over instantaneous error, with nonnegative lower bound; and
+- early residual predicts late-minus-early somatic activity with mean
+ correlation at least 0.30 and lower bound at least 0.10.
+
+The executable runner, immutable complete-grid analyzer, and entry point are
+`experiments/bci_td_confirmation.py`,
+`experiments/analyze_bci_td_confirmation.py`, and
+`experiments/bci_td_confirmation.sh`. The analyzer accepts exactly the 30
+named records. A complete pass raises the strict reviewer score from 7 to 8
+and establishes oral-B **innovation-guided plasticity**. Because `kappa=0`, it
+explicitly does not revive the failed online-control or desired-velocity
+claim. Any failed R2 gate is retained, leaves the score at 7, and closes this
+recovery without threshold repair.
diff --git a/ROADMAP.md b/ROADMAP.md
index 2e576f2..a19fa08 100644
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@ -44,6 +44,13 @@ seed, or confirmation seed is touched. The two-rate, three-task-seed R1
learning protocol and complete-grid analyzer are now committed, but its runner
is hard-gated on a complete D4 pass. Confirmation seeds remain sealed.
+The untouched recovery confirmation is now also frozen before any R1 task
+endpoint. If and only if R1 selects an eligible rate, R2 runs task seeds
+10--15 by model seeds 0--4 with task-cluster uncertainty, the original B1/B2
+signature thresholds, and a plasticity lesion. A complete R2 pass supports
+innovation-guided plasticity and raises the strict score from 7 to 8; it cannot
+support online control because the recovery fixes `kappa=0`.
+
## Frozen accept claims and gates
### C1. Innovation is necessary under naturally mixed apical traffic
diff --git a/experiments/analyze_bci_td_confirmation.py b/experiments/analyze_bci_td_confirmation.py
new file mode 100755
index 0000000..8697e38
--- /dev/null
+++ b/experiments/analyze_bci_td_confirmation.py
@@ -0,0 +1,374 @@
+#!/usr/bin/env python3
+"""Audit the frozen oral-B temporal-difference confirmation panel."""
+import argparse
+import glob
+import hashlib
+import json
+import math
+import os
+import statistics
+
+
+TASK_SEEDS = tuple(range(10, 16))
+MODEL_SEEDS = tuple(range(5))
+CONDITIONS = ("intact", "fixed_vectorizer", "plasticity_lesion", "oracle_role")
+T_CRITICAL_ONE_SIDED_95_DF5 = 2.015048373
+ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+PROTOCOL_PATH = os.path.join(ROOT, "ORAL_B_RECOVERY.md")
+RUNNER_PATH = os.path.join(ROOT, "experiments", "bci_td_confirmation.py")
+DEVELOPMENT_RUNNER_PATH = os.path.join(ROOT, "experiments", "bci_td_run.py")
+
+
+def require(condition, message):
+ if not condition:
+ raise ValueError(message)
+
+
+def sha256(path):
+ digest = hashlib.sha256()
+ with open(path, "rb") as handle:
+ for chunk in iter(lambda: handle.read(1024 * 1024), b""):
+ digest.update(chunk)
+ return digest.hexdigest()
+
+
+def finite_tree(value):
+ if isinstance(value, dict):
+ return all(finite_tree(item) for item in value.values())
+ if isinstance(value, (list, tuple)):
+ return all(finite_tree(item) for item in value)
+ if isinstance(value, (int, float)):
+ return math.isfinite(value)
+ return True
+
+
+def lower_confidence_bound(values):
+ return (statistics.mean(values)
+ - T_CRITICAL_ONE_SIDED_95_DF5
+ * statistics.stdev(values) / math.sqrt(len(values)))
+
+
+def upper_confidence_bound(values):
+ return (statistics.mean(values)
+ + T_CRITICAL_ONE_SIDED_95_DF5
+ * statistics.stdev(values) / math.sqrt(len(values)))
+
+
+def task_cluster_values(records, getter):
+ return [statistics.mean(
+ getter(records[(task_seed, model_seed)])
+ for model_seed in MODEL_SEEDS)
+ for task_seed in TASK_SEEDS]
+
+
+def summarize(values):
+ return {
+ "by_task_seed": values,
+ "mean": statistics.mean(values),
+ "one_sided_95pct_lower": lower_confidence_bound(values),
+ "one_sided_95pct_upper": upper_confidence_bound(values),
+ }
+
+
+def validate(row, path, eta, digests):
+ require(row.get("schema_version") == 1, f"{path}: schema")
+ args = row.get("args", {})
+ task_seed = args.get("task_seed")
+ model_seed = args.get("model_seed")
+ require(task_seed in TASK_SEEDS, f"{path}: task seed")
+ require(model_seed in MODEL_SEEDS, f"{path}: model seed")
+ protocol = row.get("protocol", {})
+ require(protocol.get("name") == "oral_b_td_confirmation_v1",
+ f"{path}: protocol")
+ require(protocol.get("split") == "untouched_confirmation",
+ f"{path}: split")
+ require(protocol.get("training_task_seed") == task_seed,
+ f"{path}: training seed")
+ require(protocol.get("evaluation_task_seed") == task_seed + 300_000,
+ f"{path}: evaluation seed")
+ require(protocol.get("selected_eta") == eta, f"{path}: selected eta")
+ require(protocol.get("no_further_selection") is True,
+ f"{path}: post-selection tuning")
+ require(protocol.get("confirmation_grid_size") == 30,
+ f"{path}: grid size")
+ for key in ("d4_gate", "r1_gate", "protocol"):
+ require(protocol.get(f"{key}_sha256") == digests[key],
+ f"{path}: {key} digest")
+
+ source = row.get("provenance", {})
+ require(source.get("git_tracked_dirty") is False, f"{path}: dirty")
+ require(all(source.get("tracked_inputs", {}).values())
+ and set(source.get("tracked_inputs", {})) == {
+ "runner", "development_runner", "protocol", "d4_gate", "r1_gate"},
+ f"{path}: untracked input")
+ expected_source_digests = {
+ "runner": digests["runner"],
+ "development_runner": digests["development_runner"],
+ "protocol": digests["protocol"],
+ "d4_gate": digests["d4_gate"],
+ "r1_gate": digests["r1_gate"],
+ }
+ require(source.get("input_sha256") == expected_source_digests,
+ f"{path}: source digest drift")
+ require(isinstance(source.get("git_commit"), str)
+ and len(source["git_commit"]) == 40, f"{path}: commit")
+
+ config = row.get("config", {})
+ expected = {
+ "n_plus": 5, "n_minus": 5, "n_background": 30,
+ "context_dim": 16, "steps_per_episode": 28,
+ "episodes_per_day": 64, "days": 14, "target": 0.8,
+ "inertia": 0.65, "process_noise": 0.12, "context_ar": 0.8,
+ "coupling_scale": 1.0, "predictor_eta": 0.2,
+ "vectorizer_eta": 0.03, "forward_eta": eta,
+ "perturb_sigma": 0.03, "perturb_every": 4,
+ "kappa": 0.0, "feedback": "performance_velocity",
+ }
+ for key, value in expected.items():
+ require(config.get(key) == value, f"{path}: config {key}")
+
+ require(row.get("finite") is True and finite_tree(row), f"{path}: finite")
+ for name in CONDITIONS:
+ warmup = row["warmup"][name]
+ require(warmup["batches"] == 100 and warmup["examples"] == 6400,
+ f"{path}: warmup count {name}")
+ require(warmup["instruction_present"] is False,
+ f"{path}: warmup instruction {name}")
+ require(warmup["role_cursor_scalar_observations"] == 12800,
+ f"{path}: warmup observations {name}")
+ require(warmup["predictor_max_abs_error"] <= 1e-5,
+ f"{path}: predictor {name}")
+ condition = row["conditions"][name]
+ require(len(condition["daily_success"]) == 14,
+ f"{path}: trajectory {name}")
+ cost = condition["cost"]
+ require(cost["ordinary_state_episode_steps"] == 25088,
+ f"{path}: ordinary cost {name}")
+ require(cost["online_role_perturbation_events"] == 98,
+ f"{path}: event cost {name}")
+ expected_online = 12544 if name in (
+ "intact", "plasticity_lesion") else 0
+ require(cost["conservative_online_cursor_scalar_observations"]
+ == expected_online, f"{path}: cursor cost {name}")
+ require(row["signatures"]["evaluation_episodes"] == 256,
+ f"{path}: evaluation episodes")
+
+
+def main():
+ parser = argparse.ArgumentParser()
+ parser.add_argument("--results", default="results/bci_td_confirmation")
+ parser.add_argument(
+ "--d4_gate",
+ default="results/kp_dynamic_projection_confirmation_gate.json")
+ parser.add_argument("--r1_gate", default="results/bci_td_dev_gate.json")
+ parser.add_argument("--out", default="results/bci_td_confirmation_gate.json")
+ args = parser.parse_args()
+ with open(args.d4_gate) as handle:
+ d4 = json.load(handle)
+ with open(args.r1_gate) as handle:
+ r1 = json.load(handle)
+ require(d4.get("protocol") ==
+ "kp_dynamic_neutral_projection_confirmation_v1"
+ and d4.get("status") == "passed"
+ and d4.get("review_score_after") == 7, "D4 gate")
+ require(r1.get("protocol") == "oral_b_td_development_v1"
+ and r1.get("status") == "passed"
+ and r1.get("complete_grid") is True
+ and r1.get("oral_b_confirmation_opened") is True
+ and r1.get("confirmation_seeds_touched") is False,
+ "R1 gate")
+ eta = float(r1["selected"]["eta"])
+ require(eta in (0.03, 0.1), "R1 selected eta")
+ digests = {
+ "runner": sha256(RUNNER_PATH),
+ "development_runner": sha256(DEVELOPMENT_RUNNER_PATH),
+ "protocol": sha256(PROTOCOL_PATH),
+ "d4_gate": sha256(args.d4_gate),
+ "r1_gate": sha256(args.r1_gate),
+ }
+ require(r1.get("d4_gate_sha256") == digests["d4_gate"],
+ "R1/D4 digest binding")
+
+ expected_names = {
+ f"bci_td_confirm_t{task_seed}_m{model_seed}.json"
+ for task_seed in TASK_SEEDS for model_seed in MODEL_SEEDS
+ }
+ observed_names = {
+ os.path.basename(path)
+ for path in glob.glob(os.path.join(args.results, "*.json"))
+ }
+ require(observed_names == expected_names,
+ f"confirmation grid drift: missing={sorted(expected_names-observed_names)}, "
+ f"extra={sorted(observed_names-expected_names)}")
+ records = {}
+ commits = set()
+ source_sha256 = {}
+ for task_seed in TASK_SEEDS:
+ for model_seed in MODEL_SEEDS:
+ path = os.path.join(
+ args.results,
+ f"bci_td_confirm_t{task_seed}_m{model_seed}.json")
+ with open(path) as handle:
+ row = json.load(handle)
+ validate(row, path, eta, digests)
+ records[(task_seed, model_seed)] = row
+ commits.add(row["provenance"]["git_commit"])
+ source_sha256[path] = sha256(path)
+ require(len(commits) == 1, "R2 source revision drift")
+
+ metrics = {}
+ getters = {
+ "intact_final": lambda r: r["conditions"]["intact"]["final_success"],
+ "intact_gain": lambda r: r["conditions"]["intact"]["learning_gain"],
+ "fixed_final_gap": lambda r: (
+ r["conditions"]["intact"]["final_success"]
+ - r["conditions"]["fixed_vectorizer"]["final_success"]),
+ "oracle_final_deficit": lambda r: (
+ r["conditions"]["oracle_role"]["final_success"]
+ - r["conditions"]["intact"]["final_success"]),
+ "plasticity_half_margin": lambda r: (
+ 0.5 * r["conditions"]["intact"]["learning_gain"]
+ - r["conditions"]["plasticity_lesion"]["learning_gain"]),
+ "role_cosine": lambda r: r["conditions"]["intact"][
+ "role_cosine_after_training"],
+ "residual_soma_corr": lambda r: r["signatures"][
+ "mean_abs_residual_soma_corr"],
+ "raw_residual_corr_gap": lambda r: r["signatures"][
+ "raw_minus_residual_abs_soma_corr"],
+ "surrounding_event_accuracy": lambda r: r["signatures"][
+ "surrounding_event_decoder_balanced_acc"],
+ "decoder_distance_corr": lambda r: r["signatures"][
+ "decoder_distance_residual_corr"],
+ "residual_outcome_accuracy": lambda r: r["signatures"][
+ "residual_outcome_decoder_balanced_acc"],
+ "residual_soma_outcome_gap": lambda r: r["signatures"][
+ "residual_minus_soma_outcome_acc"],
+ "sign_inversion": lambda r: r["signatures"][
+ "causal_role_sign_inversion_index"],
+ "velocity_advantage": lambda r: r["signatures"][
+ "velocity_minus_error_abs_cv_corr"],
+ "longitudinal_prediction": lambda r: r["signatures"][
+ "early_residual_late_activity_change_corr"],
+ }
+ clustered = {}
+ for name, getter in getters.items():
+ values = task_cluster_values(records, getter)
+ clustered[name] = values
+ metrics[name] = summarize(values)
+
+ all_role_cosines = [getters["role_cosine"](row)
+ for row in records.values()]
+ all_signs = [getters["sign_inversion"](row)
+ for row in records.values()]
+ learning_checks = {
+ "mean_intact_final_at_least_0p70":
+ metrics["intact_final"]["mean"] >= 0.70,
+ "every_task_mean_final_at_least_0p60":
+ min(clustered["intact_final"]) >= 0.60,
+ "intact_final_lower_bound_at_least_0p60":
+ metrics["intact_final"]["one_sided_95pct_lower"] >= 0.60,
+ "mean_learning_gain_at_least_0p10":
+ metrics["intact_gain"]["mean"] >= 0.10,
+ "learning_gain_lower_bound_at_least_0p05":
+ metrics["intact_gain"]["one_sided_95pct_lower"] >= 0.05,
+ "mean_fixed_gap_at_least_0p20":
+ metrics["fixed_final_gap"]["mean"] >= 0.20,
+ "fixed_gap_lower_bound_at_least_0p10":
+ metrics["fixed_final_gap"]["one_sided_95pct_lower"] >= 0.10,
+ "mean_oracle_deficit_at_most_0p10":
+ metrics["oracle_final_deficit"]["mean"] <= 0.10,
+ "oracle_deficit_upper_bound_at_most_0p20":
+ metrics["oracle_final_deficit"]["one_sided_95pct_upper"] <= 0.20,
+ "plasticity_lesion_half_margin_lower_bound_nonnegative":
+ metrics["plasticity_half_margin"]["one_sided_95pct_lower"] >= 0.0,
+ "mean_role_cosine_at_least_0p80":
+ metrics["role_cosine"]["mean"] >= 0.80,
+ "every_role_cosine_at_least_0p70": min(all_role_cosines) >= 0.70,
+ }
+ innovation_checks = {
+ "mean_residual_soma_corr_at_most_0p10":
+ metrics["residual_soma_corr"]["mean"] <= 0.10,
+ "residual_soma_corr_upper_bound_at_most_0p12":
+ metrics["residual_soma_corr"]["one_sided_95pct_upper"] <= 0.12,
+ "mean_raw_residual_corr_gap_at_least_0p20":
+ metrics["raw_residual_corr_gap"]["mean"] >= 0.20,
+ "raw_residual_corr_gap_lower_bound_at_least_0p15":
+ metrics["raw_residual_corr_gap"]["one_sided_95pct_lower"] >= 0.15,
+ "mean_surrounding_event_accuracy_at_least_0p55":
+ metrics["surrounding_event_accuracy"]["mean"] >= 0.55,
+ "surrounding_event_accuracy_lower_bound_at_least_0p52":
+ metrics["surrounding_event_accuracy"]["one_sided_95pct_lower"] >= 0.52,
+ "mean_decoder_distance_corr_at_least_0p10":
+ metrics["decoder_distance_corr"]["mean"] >= 0.10,
+ "decoder_distance_corr_lower_bound_at_least_0p02":
+ metrics["decoder_distance_corr"]["one_sided_95pct_lower"] >= 0.02,
+ }
+ vectorization_checks = {
+ "mean_residual_outcome_accuracy_at_least_0p57":
+ metrics["residual_outcome_accuracy"]["mean"] >= 0.57,
+ "residual_outcome_accuracy_lower_bound_at_least_0p53":
+ metrics["residual_outcome_accuracy"]["one_sided_95pct_lower"] >= 0.53,
+ "mean_residual_soma_outcome_gap_at_least_0p03":
+ metrics["residual_soma_outcome_gap"]["mean"] >= 0.03,
+ "residual_soma_outcome_gap_lower_bound_nonnegative":
+ metrics["residual_soma_outcome_gap"]["one_sided_95pct_lower"] >= 0.0,
+ "positive_sign_inversion_in_at_least_25_of_30":
+ sum(value > 0 for value in all_signs) >= 25,
+ "positive_sign_inversion_in_every_task_cluster":
+ min(clustered["sign_inversion"]) > 0.0,
+ "mean_velocity_advantage_at_least_0p05":
+ metrics["velocity_advantage"]["mean"] >= 0.05,
+ "velocity_advantage_lower_bound_nonnegative":
+ metrics["velocity_advantage"]["one_sided_95pct_lower"] >= 0.0,
+ "mean_longitudinal_prediction_at_least_0p30":
+ metrics["longitudinal_prediction"]["mean"] >= 0.30,
+ "longitudinal_prediction_lower_bound_at_least_0p10":
+ metrics["longitudinal_prediction"]["one_sided_95pct_lower"] >= 0.10,
+ }
+ checks = {
+ "all_records_finite_paired_and_cost_audited": True,
+ "learning_and_plasticity": learning_checks,
+ "innovation_identification_and_network_prediction": innovation_checks,
+ "outcome_and_causal_role_vectorization": vectorization_checks,
+ }
+ passed = all(
+ value
+ for category, values in checks.items()
+ for value in ([values] if isinstance(values, bool) else values.values()))
+ output = {
+ "protocol": "oral_b_td_confirmation_v1",
+ "status": "passed" if passed else "failed",
+ "complete_grid": True,
+ "selected_eta": eta,
+ "checks": checks,
+ "metrics": metrics,
+ "positive_sign_count": sum(value > 0 for value in all_signs),
+ "source_commit": next(iter(commits)),
+ "source_sha256": source_sha256,
+ "d4_gate_sha256": digests["d4_gate"],
+ "r1_gate_sha256": digests["r1_gate"],
+ "protocol_sha256": digests["protocol"],
+ "oral_b_plasticity_innovation_established": passed,
+ "online_control_or_desired_velocity_established": False,
+ "review_score_before": 7,
+ "review_score_after": 8 if passed else 7,
+ "score_change_rule": (
+ "only a complete untouched R2 pass establishes oral-B "
+ "innovation-guided plasticity; kappa=0 cannot establish online control"),
+ }
+ os.makedirs(os.path.dirname(os.path.abspath(args.out)), exist_ok=True)
+ if os.path.exists(args.out):
+ with open(args.out) as handle:
+ existing = json.load(handle)
+ require(existing == output,
+ "existing R2 gate differs from deterministic re-audit")
+ print(json.dumps(output, indent=2))
+ return
+ with open(args.out, "w") as handle:
+ json.dump(output, handle, indent=2, sort_keys=True)
+ handle.write("\n")
+ print(json.dumps(output, indent=2))
+
+
+if __name__ == "__main__":
+ main()
diff --git a/experiments/bci_td_confirmation.py b/experiments/bci_td_confirmation.py
new file mode 100755
index 0000000..2a5590d
--- /dev/null
+++ b/experiments/bci_td_confirmation.py
@@ -0,0 +1,199 @@
+#!/usr/bin/env python3
+"""Run one cell of the frozen oral-B temporal-difference confirmation."""
+import argparse
+import hashlib
+import json
+import math
+import os
+import platform
+import resource
+import subprocess
+import sys
+import time
+
+import torch
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+from experiments.bci_td_run import (CONDITIONS, build_config, evaluate,
+ neutral_warmup, train)
+from sdil.bci import BCISDIL, generate_trajectories
+from sdil.bci_metrics import signature_metrics
+
+
+ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+PROTOCOL_PATH = os.path.join(ROOT, "ORAL_B_RECOVERY.md")
+TASK_SEEDS = tuple(range(10, 16))
+MODEL_SEEDS = tuple(range(5))
+
+
+def sha256(path):
+ digest = hashlib.sha256()
+ with open(path, "rb") as handle:
+ for chunk in iter(lambda: handle.read(1024 * 1024), b""):
+ digest.update(chunk)
+ return digest.hexdigest()
+
+
+def finite_tree(value):
+ if isinstance(value, dict):
+ return all(finite_tree(item) for item in value.values())
+ if isinstance(value, (list, tuple)):
+ return all(finite_tree(item) for item in value)
+ if isinstance(value, (int, float)):
+ return math.isfinite(value)
+ return True
+
+
+def provenance(d4_gate, r1_gate):
+ def run(command):
+ return subprocess.run(
+ command, cwd=ROOT, check=True, capture_output=True,
+ text=True).stdout.strip()
+
+ paths = {
+ "runner": os.path.abspath(__file__),
+ "development_runner": os.path.join(ROOT, "experiments", "bci_td_run.py"),
+ "protocol": PROTOCOL_PATH,
+ "d4_gate": os.path.abspath(d4_gate),
+ "r1_gate": os.path.abspath(r1_gate),
+ }
+ relative = {name: os.path.relpath(path, ROOT)
+ for name, path in paths.items()}
+ tracked = {
+ name: subprocess.run(
+ ["git", "ls-files", "--error-unmatch", path], cwd=ROOT,
+ capture_output=True).returncode == 0
+ for name, path in relative.items()
+ }
+ return {
+ "git_commit": run(["git", "rev-parse", "HEAD"]),
+ "git_tracked_dirty": bool(run(
+ ["git", "status", "--porcelain", "--untracked-files=no"])),
+ "tracked_inputs": tracked,
+ "input_sha256": {name: sha256(path) for name, path in paths.items()},
+ }
+
+
+def require_gate(d4, r1, d4_digest):
+ if not (d4.get("protocol") ==
+ "kp_dynamic_neutral_projection_confirmation_v1"
+ and d4.get("status") == "passed"
+ and d4.get("review_score_after") == 7):
+ raise ValueError("oral-B R2 requires the complete audited D4 pass")
+ if not (r1.get("protocol") == "oral_b_td_development_v1"
+ and r1.get("status") == "passed"
+ and r1.get("complete_grid") is True
+ and r1.get("confirmation_seeds_touched") is False
+ and r1.get("oral_b_confirmation_opened") is True
+ and r1.get("review_score_after") == 7
+ and r1.get("d4_gate_sha256") == d4_digest
+ and r1.get("selected", {}).get("eta") in (0.03, 0.1)):
+ raise ValueError("oral-B R2 requires the complete eligible R1 gate")
+
+
+def main():
+ parser = argparse.ArgumentParser()
+ parser.add_argument(
+ "--d4_gate",
+ default="results/kp_dynamic_projection_confirmation_gate.json")
+ parser.add_argument(
+ "--r1_gate", default="results/bci_td_dev_gate.json")
+ parser.add_argument("--task-seed", type=int, choices=TASK_SEEDS, required=True)
+ parser.add_argument("--model-seed", type=int, choices=MODEL_SEEDS, required=True)
+ parser.add_argument("--outdir", default="results/bci_td_confirmation")
+ args = parser.parse_args()
+
+ with open(args.d4_gate) as handle:
+ d4 = json.load(handle)
+ with open(args.r1_gate) as handle:
+ r1 = json.load(handle)
+ d4_digest = sha256(args.d4_gate)
+ require_gate(d4, r1, d4_digest)
+ eta = float(r1["selected"]["eta"])
+ source = provenance(args.d4_gate, args.r1_gate)
+ if (source["git_tracked_dirty"]
+ or not all(source["tracked_inputs"].values())):
+ raise RuntimeError(
+ "R2 requires clean tracked code, protocol, D4 gate, and R1 gate")
+
+ cfg = build_config(eta)
+ trajectories = generate_trajectories(cfg, args.task_seed)
+ evaluation_seed = args.task_seed + 300_000
+ evaluation = generate_trajectories(
+ cfg, evaluation_seed, days=1, episodes=256)
+ initial = BCISDIL(cfg, args.model_seed)
+ conditions = {}
+ trained = {}
+ warmups = {}
+ training_events = None
+ started = time.perf_counter()
+ for name in CONDITIONS:
+ model, warmup = neutral_warmup(
+ initial, args.task_seed, args.model_seed, name)
+ events, report = train(
+ model, trajectories, name, collect=name == "intact")
+ warmups[name] = warmup
+ trained[name] = model
+ conditions[name] = report
+ if name == "intact":
+ training_events = events
+
+ evaluation_events = None
+ for name in CONDITIONS:
+ report = evaluate(
+ trained[name], evaluation, collect=name == "intact")
+ conditions[name]["final_success"] = report["success_rate"]
+ if name == "intact":
+ evaluation_events = report["events"]
+ signatures = signature_metrics(
+ training_events, evaluation_events, cfg, trained["intact"].role)
+
+ result = {
+ "schema_version": 1,
+ "protocol": {
+ "name": "oral_b_td_confirmation_v1",
+ "split": "untouched_confirmation",
+ "training_task_seed": args.task_seed,
+ "evaluation_task_seed": evaluation_seed,
+ "selected_eta": eta,
+ "no_further_selection": True,
+ "confirmation_grid_size": len(TASK_SEEDS) * len(MODEL_SEEDS),
+ "d4_gate_sha256": source["input_sha256"]["d4_gate"],
+ "r1_gate_sha256": source["input_sha256"]["r1_gate"],
+ "protocol_sha256": source["input_sha256"]["protocol"],
+ },
+ "args": vars(args),
+ "config": vars(cfg),
+ "provenance": source,
+ "warmup": warmups,
+ "conditions": conditions,
+ "signatures": signatures,
+ "wall_s": time.perf_counter() - started,
+ "peak_rss_mib": resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024,
+ "hardware": {
+ "device": "cpu", "platform": platform.platform(),
+ "torch_version": torch.__version__, "threads": torch.get_num_threads(),
+ },
+ }
+ result["finite"] = finite_tree(result)
+ if not result["finite"]:
+ raise RuntimeError("non-finite oral-B R2 record")
+ os.makedirs(args.outdir, exist_ok=True)
+ path = os.path.join(
+ args.outdir,
+ f"bci_td_confirm_t{args.task_seed}_m{args.model_seed}.json")
+ if os.path.exists(path):
+ raise FileExistsError(f"refusing to overwrite {path}")
+ with open(path, "w") as handle:
+ json.dump(result, handle, indent=2, sort_keys=True)
+ handle.write("\n")
+ print(json.dumps({
+ "path": path,
+ "finite": result["finite"],
+ "intact_final": conditions["intact"]["final_success"],
+ "sign_inversion": signatures["causal_role_sign_inversion_index"],
+ }, indent=2))
+
+
+if __name__ == "__main__":
+ main()
diff --git a/experiments/bci_td_confirmation.sh b/experiments/bci_td_confirmation.sh
new file mode 100755
index 0000000..aab8c8c
--- /dev/null
+++ b/experiments/bci_td_confirmation.sh
@@ -0,0 +1,16 @@
+#!/usr/bin/env bash
+# Frozen R1-gated oral-B confirmation: 6 task seeds x 5 model seeds.
+set -eu
+
+cd "$(dirname "$0")/.."
+PY=/home/yurenh2/miniconda3/envs/ep_pascal/bin/python3
+
+for task_seed in 10 11 12 13 14 15; do
+ for model_seed in 0 1 2 3 4; do
+ OMP_NUM_THREADS=1 MKL_NUM_THREADS=1 "$PY" \
+ experiments/bci_td_confirmation.py \
+ --task-seed "$task_seed" --model-seed "$model_seed"
+ done
+done
+
+"$PY" experiments/analyze_bci_td_confirmation.py
diff --git a/experiments/finalize_accept.sh b/experiments/finalize_accept.sh
index d727f2e..d2aa4cc 100755
--- a/experiments/finalize_accept.sh
+++ b/experiments/finalize_accept.sh
@@ -13,7 +13,9 @@ experiments/finalize_claims.sh
/home/yurenh2/miniconda3/envs/ep_pascal/bin/python3 \
experiments/bci_smoke.py
/home/yurenh2/miniconda3/envs/ep_pascal/bin/python3 -m py_compile \
- experiments/bci_td_run.py experiments/analyze_bci_td_development.py
+ experiments/bci_td_run.py experiments/analyze_bci_td_development.py \
+ experiments/bci_td_confirmation.py \
+ experiments/analyze_bci_td_confirmation.py
/home/yurenh2/miniconda3/envs/ep_pascal/bin/python3 \
experiments/analyze_kp_dynamic_projection.py >/dev/null
/home/yurenh2/miniconda3/envs/ep_pascal/bin/python3 \