diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 20:41:53 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 20:41:53 -0500 |
| commit | 18536203bb1cc7f313f8f3ed2767f128e5407c3b (patch) | |
| tree | c52636127ce2a638a0c42832295fb8ff3c080195 | |
| parent | 0bfaef245b13d0e2a0d0918c74789dc0238ded74 (diff) | |
protocol: freeze dynamic standard-depth recovery
| -rw-r--r-- | ORAL_A_RECOVERY.md | 91 | ||||
| -rw-r--r-- | ROADMAP.md | 8 | ||||
| -rwxr-xr-x | experiments/analyze_oral_a_dynamic_scaling.py | 418 | ||||
| -rwxr-xr-x | experiments/finalize_accept.sh | 8 | ||||
| -rwxr-xr-x | experiments/oral_a_dynamic_scaling.py | 187 |
5 files changed, 711 insertions, 1 deletions
diff --git a/ORAL_A_RECOVERY.md b/ORAL_A_RECOVERY.md new file mode 100644 index 0000000..55189b1 --- /dev/null +++ b/ORAL_A_RECOVERY.md @@ -0,0 +1,91 @@ +# Oral-A recovery: dynamic innovation on standard ResNet depth + +This branch does not reopen the failed learned-vectorizer A3/A4 protocol in +`ORAL_A.md`. It asks a different question that became available only after the +reciprocal Kolen--Pollack substrate and fast instruction-off neutral projection +were established: does load-bearing somato-dendritic innovation retain +standard-image performance, credit alignment, and a positive depth benefit as +the same canonical CIFAR ResNet family grows? + +No recovery task endpoint may be generated until this protocol and its +executable gate are committed. Execution is hard-gated on both a complete D4 +accept pass (`7/10`) and a complete oral-B R2 pass (`8/10`). Thus the required +order remains accept bar, biological oral-B, then standard-depth oral-A. + +## Fixed panel and reuse boundary + +Use CIFAR-10 ResNet-20/32/56 with width 16, BatchNorm, option-A shortcuts, +batch size 128, all 50,000 training examples, 200 epochs, SGD momentum 0.9, +weight decay `1e-4`, base learning rate 0.1 with 10-fold drops at epochs 100 +and 150, no warmup, and model/data-loader seeds 10--14. Test is evaluated +exactly once at the endpoint; validation and intermediate test evaluations are +zero. + +The matched methods are: + +1. exact BP on the identical forward architecture; +2. tuned DFA at its already selected hidden rate 0.03 and output rate 0.1; +3. clean reciprocal Kolen--Pollack at hidden/output rate 0.1; and +4. dynamic innovation on the same reciprocal substrate, with traffic ratio 4, + traffic seed `5000 + model_seed`, 64 instruction-off calibration examples, + one closed-form neutral projection, and no later predictor update. + +The already frozen D4 clean-KP and dynamic records at ResNet-20/seeds 10--14 +are reused verbatim rather than rerun. The recovery therefore adds exactly 50 +records: BP and DFA at depth 20, and all four methods at depths 32 and 56. +Together with D4 this forms an exact 60-cell panel. All new records must share +one clean tracked source revision; all D4 records retain their separate frozen +source revision. No completed record may be deleted, replaced, or pooled with +a pilot. + +Fixed HFA remains closed after its preregistered short gate failed, while FF, +PEPITA, canonical EP, BurstCCN, and Dual Prop retain their method-native +audits. They are reported as separate baselines rather than silently moved +onto this matched standard-ResNet grid. + +## Frozen mechanism and cost invariants + +Every dynamic record must retain all D4 invariants: finite 200-epoch +trajectory; mean early-third credit alignment measured only diagnostically; +final feedback/forward cosine at least 0.98 and epoch-151--200 mean at least +0.97; instruction-to-teaching RMS error at most `1e-4`; maximum post-projection +traffic ratio and absolute soma slope at most `1e-5`; zero instruction leakage +into neutral observations; exactly 64 predictor-update examples; exactly one +neutral observation per ordinary example; and zero task-loss queries. + +At every depth, dynamic and clean KP must each use at most `1.34x` the matched +BP MAC estimate. Dynamic elementwise projection work is reported separately +and must be positive. Peak allocated memory may not exceed 8 GiB on the +authorized GTX 1080 devices. Wall time is reported but is never substituted +for MACs, queries, or memory. + +## Frozen oral-A gate + +The five model/data seeds are the independent paired units. One-sided 95% +Student-t bounds use four degrees of freedom. Oral-A passes only if every +record and every audited value is finite and all of the following hold: + +1. Mean BP accuracy is at least 90% at every depth and every dynamic endpoint + is at least 87%, preventing joint collapse from satisfying a relative gate. +2. At each depth, mean dynamic accuracy is within 2 points of matched BP and + the one-sided 95% upper bound on the paired deficit is at most 3 points. +3. At each depth, mean dynamic accuracy is within 1.5 points of clean KP and + the upper bound on that paired deficit is at most 2.5 points. +4. At depth 56, dynamic exceeds tuned DFA by at least 2 points on average and + the one-sided lower bound on the paired advantage is at least 1 point. +5. Dynamic gains at least 0.5 points from depth 20 to 56 on average, its + one-sided lower bound is nonnegative, and at least four of five paired + seeds improve. Its mean depth gain must also be no more than 1 point below + BP's paired depth gain. +6. Mean dynamic early-third alignment is at least 0.85 at every depth; every + depth-56 seed is at least 0.80; and the depth-56 mean retains at least 90% + of the depth-20 mean. +7. Every mechanism, query, MAC, elementwise-work, memory, source, split, and + evaluation invariant above passes without exception. + +The executable runner and immutable complete-grid analyzer are +`experiments/oral_a_dynamic_scaling.py` and +`experiments/analyze_oral_a_dynamic_scaling.py`. A complete pass raises the +strict reviewer score from 8 to 9 and establishes oral-A standard-depth +scaling for dynamic innovation. Any failure is retained, leaves the score at +8, and closes this branch without depth, seed, optimizer, or threshold repair. @@ -366,6 +366,14 @@ passed. Per the stop rule, the five-seed ResNet-20/32/56 test panel was never run. The exact BatchNorm/local-gradient mechanics remain verified, but the current training recipe does not establish standard-network scaling. +A separate dynamic-innovation recovery is now frozen in +`ORAL_A_RECOVERY.md` before any new endpoint. It does not reopen failed A3/A4: +only complete D4 and oral-B R2 passes can unlock a 60-cell +BP/DFA/clean-KP/dynamic ResNet-20/32/56 panel. D4 supplies the ten depth-20 +KP/dynamic cells verbatim; exactly 50 new cells are permitted. A full pass is +the sole 8-to-9 score gate and must show a positive paired depth benefit, not +merely survival on another depth-flat task. + Post-failure diagnosis is recorded separately in `results/oral_a_failure_diagnosis.json`: prediction--target cosine remains below `8.51e-5` in magnitude, calibration MSE is indistinguishable from target diff --git a/experiments/analyze_oral_a_dynamic_scaling.py b/experiments/analyze_oral_a_dynamic_scaling.py new file mode 100755 index 0000000..ccd4fa9 --- /dev/null +++ b/experiments/analyze_oral_a_dynamic_scaling.py @@ -0,0 +1,418 @@ +#!/usr/bin/env python3 +"""Audit the frozen standard-depth dynamic-innovation recovery panel.""" +import argparse +import glob +import hashlib +import json +import math +import os +import statistics + + +DEPTHS = (20, 32, 56) +SEEDS = tuple(range(10, 15)) +METHODS = ("bp", "dfa", "clean_kp", "dynamic") +T_CRITICAL_ONE_SIDED_95_DF4 = 2.131846786 +ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +PROTOCOL_PATH = os.path.join(ROOT, "ORAL_A_RECOVERY.md") +RUNNER_PATH = os.path.join(ROOT, "experiments", "oral_a_dynamic_scaling.py") + + +def require(condition, message): + if not condition: + raise ValueError(message) + + +def sha256(path): + digest = hashlib.sha256() + with open(path, "rb") as handle: + for chunk in iter(lambda: handle.read(1024 * 1024), b""): + digest.update(chunk) + return digest.hexdigest() + + +def numeric_leaves(value): + if isinstance(value, bool) or value is None: + return + if isinstance(value, (int, float)): + yield float(value) + elif isinstance(value, dict): + for child in value.values(): + yield from numeric_leaves(child) + elif isinstance(value, (list, tuple)): + for child in value: + yield from numeric_leaves(child) + + +def upper_confidence_bound(values): + return (statistics.mean(values) + + T_CRITICAL_ONE_SIDED_95_DF4 + * statistics.stdev(values) / math.sqrt(len(values))) + + +def lower_confidence_bound(values): + return (statistics.mean(values) + - T_CRITICAL_ONE_SIDED_95_DF4 + * statistics.stdev(values) / math.sqrt(len(values))) + + +def expected_args(method, depth, seed): + common = { + "depth": depth, "width": 16, "seed": seed, "loader_seed": seed, + "batch_size": 128, "epochs": 200, "train_limit": 0, + "val_examples": 0, "split_seed": 2027, "eval_split": "test", + "eval_every": 0, "augment_train": 1, "lr_schedule": "step", + "lr_milestones": "100,150", "lr_gamma": 0.1, + "warmup_epochs": 0, "momentum": 0.9, "weight_decay": 1e-4, + "normalization": "batchnorm", "a_scale": 1.0, + } + if method == "bp": + return {**common, "mode": "bp", "lr": 0.1, "output_lr": None, + "alignment_probe": 0} + if method == "dfa": + return {**common, "mode": "dfa", "lr": 0.03, "output_lr": 0.1, + "vectorizer_mode": "spatial_template", "alignment_probe": 32} + if method == "clean_kp": + return {**common, "mode": "kp", "lr": 0.1, "output_lr": 0.1, + "alignment_probe": 32} + return { + **common, "mode": "kp_traffic", "lr": 0.1, "output_lr": 0.1, + "alignment_probe": 32, "traffic_rule": "innovation", + "predictor_mode": "closed_form", "neutral_projection": 1, + "traffic_seed": 5000 + seed, "traffic_ratio": 4.0, + "traffic_calibration_examples": 64, "learn_P": 1, "eta_P": 0.1, + "predictor_warmup_steps": 1, "predictor_every": 0, + } + + +def validate_record(record, path, method, depth, seed): + for key, value in expected_args(method, depth, seed).items(): + require(record.get("args", {}).get(key) == value, + f"{path}: argument drift {key}") + require(record.get("provenance", {}).get("git_tracked_dirty") is False, + f"{path}: dirty source") + split = record.get("split", {}) + require(split.get("train_examples") == 50_000 + and split.get("validation_examples") == 0 + and split.get("validation_index_sha256") is None + and split.get("test_examples") == 10_000, + f"{path}: split drift") + evaluation = record.get("evaluation_protocol", {}) + require(evaluation.get("validation_evaluations") == 0 + and evaluation.get("test_evaluations") == 1 + and evaluation.get("test_used_for_selection") is False, + f"{path}: evaluation drift") + epochs = record.get("epochs", []) + require(len(epochs) == 200 + and all(row.get("epoch") == index + 1 + for index, row in enumerate(epochs)), + f"{path}: trajectory drift") + values = list(numeric_leaves({ + "final": record.get("final"), "epochs": epochs, + "diagnostics": record.get("diagnostics"), + "work": record.get("work"), "hardware": record.get("hardware"), + })) + require(record.get("final", {}).get("finite") is True + and all(math.isfinite(value) for value in values), + f"{path}: nonfinite") + if method == "bp": + require(record.get("diagnostics") is None, f"{path}: BP diagnostic") + else: + require(record.get("diagnostics") is not None, + f"{path}: missing local diagnostic") + return { + "accuracy": float(record["final"]["accuracy"]), + "loss": float(record["final"]["loss"]), + "early_alignment": (None if method == "bp" else float( + record["diagnostics"]["early_third_mean"])), + "total_macs": int(record["work"]["total_macs_estimate"]), + "elementwise_operations": int(record["work"][ + "elementwise_operations_estimate"]), + "logical_queries": int(record["work"]["logical_batch_loss_queries"]), + "peak_memory": int(record["hardware"][ + "peak_memory_allocated_bytes"]), + "source_commit": record["provenance"]["git_commit"], + } + + +def dynamic_invariants(record, depth, seed, failures): + epochs = record["epochs"] + projection = [row.get("neutral_projection") for row in epochs] + mixed = [row.get("mixed_apical") for row in epochs] + tracking = [row.get("feedback_tracking") for row in epochs] + tag = f"dynamic_d{depth}_s{seed}" + if any(value is None for value in projection + mixed + tracking): + failures.append(f"{tag}:mechanism_trajectory") + return {} + signal_error = max(abs( + float(values["teaching_rms"]) + / max(float(values["instruction_rms"]), 1e-30) - 1.0) + for values in mixed) + post_ratio = max(float(value[ + "maximum_post_projection_traffic_rms_ratio"]) for value in projection) + post_slope = max(float(value[ + "maximum_absolute_post_projection_soma_slope"]) for value in projection) + instruction_observations = sum(int( + value["instruction_observations"]) for value in projection) + early = float(record["diagnostics"]["early_third_mean"]) + final_feedback = float(record["diagnostics"][ + "mean_feedback_forward_cosine"]) + late_feedback = statistics.mean(float(value[ + "mean_feedback_forward_cosine"]) for value in tracking[150:]) + counters = record["counters"] + work = record["work"] + warmup = record.get("predictor_warmup", {}) + if not (warmup.get("mode") == "closed_form" + and warmup.get("steps") == 1 + and warmup.get("examples") == 64 + and warmup.get("instruction_present") is False + and warmup.get("task_loader_state_restored") is True + and warmup.get("reuses_traffic_calibration_forward") is True): + failures.append(f"{tag}:warmup") + if signal_error > 1e-4: + failures.append(f"{tag}:instruction_ratio") + if post_ratio > 1e-5: + failures.append(f"{tag}:post_ratio") + if post_slope > 1e-5: + failures.append(f"{tag}:post_slope") + if instruction_observations != 0: + failures.append(f"{tag}:instruction_leak") + if not (counters["predictor_update_examples"] == 64 + and counters["predictor_warmup_examples"] == 0 + and counters["neutral_projection_examples"] + == counters["ordinary_examples"] == 10_000_000): + failures.append(f"{tag}:observation_counts") + if final_feedback < 0.98 or late_feedback < 0.97: + failures.append(f"{tag}:feedback_tracking") + if work["logical_batch_loss_queries"] != 0: + failures.append(f"{tag}:queries") + if not (work["elementwise_operations_estimate"] > 0 + and work["neutral_projection_observations"] == 10_000_000): + failures.append(f"{tag}:elementwise_work") + return { + "instruction_ratio_error": signal_error, + "post_projection_traffic_ratio": post_ratio, + "post_projection_soma_slope": post_slope, + "early_alignment": early, + "final_feedback_cosine": final_feedback, + "last50_feedback_cosine": late_feedback, + } + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument( + "--d4_results", default="results/kp_dynamic_projection_confirmation") + parser.add_argument( + "--new_results", default="results/oral_a_dynamic_scaling") + parser.add_argument( + "--d4_gate", + default="results/kp_dynamic_projection_confirmation_gate.json") + parser.add_argument( + "--r2_gate", default="results/bci_td_confirmation_gate.json") + parser.add_argument( + "--out", default="results/oral_a_dynamic_scaling_gate.json") + args = parser.parse_args() + with open(args.d4_gate) as handle: + d4 = json.load(handle) + with open(args.r2_gate) as handle: + r2 = json.load(handle) + d4_digest = sha256(args.d4_gate) + require(d4.get("protocol") == + "kp_dynamic_neutral_projection_confirmation_v1" + and d4.get("status") == "passed" + and d4.get("review_score_after") == 7, "D4 gate") + require(r2.get("protocol") == "oral_b_td_confirmation_v1" + and r2.get("status") == "passed" + and r2.get("oral_b_plasticity_innovation_established") is True + and r2.get("review_score_after") == 8 + and r2.get("d4_gate_sha256") == d4_digest, "R2 gate") + + expected_d4 = { + f"seed{seed}_{condition}.json" + for seed in SEEDS for condition in ("clean_kp", "dynamic") + } + observed_d4 = {os.path.basename(path) for path in glob.glob( + os.path.join(args.d4_results, "*.json"))} + require(observed_d4 == expected_d4, + f"D4 record drift: missing={sorted(expected_d4-observed_d4)}, " + f"extra={sorted(observed_d4-expected_d4)}") + expected_new = { + f"{method}_d{depth}_s{seed}.json" + for depth in DEPTHS for seed in SEEDS for method in METHODS + if not (depth == 20 and method in ("clean_kp", "dynamic")) + } + observed_new = {os.path.basename(path) for path in glob.glob( + os.path.join(args.new_results, "*.json"))} + require(observed_new == expected_new, + f"new record drift: missing={sorted(expected_new-observed_new)}, " + f"extra={sorted(observed_new-expected_new)}") + + records = {} + rows = {} + source_sha256 = {} + d4_commits = set() + new_commits = set() + for seed in SEEDS: + for method, condition in (("clean_kp", "clean_kp"), + ("dynamic", "dynamic")): + path = os.path.join(args.d4_results, f"seed{seed}_{condition}.json") + with open(path) as handle: + record = json.load(handle) + records[(method, 20, seed)] = record + rows[(method, 20, seed)] = validate_record( + record, path, method, 20, seed) + d4_commits.add(rows[(method, 20, seed)]["source_commit"]) + source_sha256[path] = sha256(path) + for depth in DEPTHS: + for seed in SEEDS: + for method in METHODS: + if depth == 20 and method in ("clean_kp", "dynamic"): + continue + path = os.path.join( + args.new_results, f"{method}_d{depth}_s{seed}.json") + with open(path) as handle: + record = json.load(handle) + records[(method, depth, seed)] = record + rows[(method, depth, seed)] = validate_record( + record, path, method, depth, seed) + new_commits.add(rows[(method, depth, seed)]["source_commit"]) + source_sha256[path] = sha256(path) + require(len(rows) == 60, "oral-A grid must contain exactly 60 records") + require(len(d4_commits) == 1 and len(new_commits) == 1, + "source revision drift") + require(next(iter(d4_commits)) == d4["metrics"]["source_commit"], + "D4 source does not match its gate") + + failures = [] + mechanism = {} + for depth in DEPTHS: + for seed in SEEDS: + dynamic = records[("dynamic", depth, seed)] + mechanism[f"d{depth}_s{seed}"] = dynamic_invariants( + dynamic, depth, seed, failures) + bp_macs = rows[("bp", depth, seed)]["total_macs"] + for method in ("clean_kp", "dynamic"): + if rows[(method, depth, seed)]["total_macs"] > 1.34 * bp_macs: + failures.append(f"{method}_d{depth}_s{seed}:mac_cost") + if rows[("dynamic", depth, seed)]["peak_memory"] > 8 * 1024 ** 3: + failures.append(f"dynamic_d{depth}_s{seed}:peak_memory") + for method in ("dfa", "clean_kp", "dynamic"): + if rows[(method, depth, seed)]["logical_queries"] != 0: + failures.append(f"{method}_d{depth}_s{seed}:queries") + + accuracies = { + method: {depth: [rows[(method, depth, seed)]["accuracy"] + for seed in SEEDS] + for depth in DEPTHS} + for method in METHODS + } + alignments = { + depth: [rows[("dynamic", depth, seed)]["early_alignment"] + for seed in SEEDS] + for depth in DEPTHS + } + bp_deficits = {depth: [bp - dynamic for bp, dynamic in zip( + accuracies["bp"][depth], accuracies["dynamic"][depth])] + for depth in DEPTHS} + kp_deficits = {depth: [kp - dynamic for kp, dynamic in zip( + accuracies["clean_kp"][depth], accuracies["dynamic"][depth])] + for depth in DEPTHS} + d56_dfa_advantage = [dynamic - dfa for dynamic, dfa in zip( + accuracies["dynamic"][56], accuracies["dfa"][56])] + dynamic_depth_gain = [deep - shallow for shallow, deep in zip( + accuracies["dynamic"][20], accuracies["dynamic"][56])] + bp_depth_gain = [deep - shallow for shallow, deep in zip( + accuracies["bp"][20], accuracies["bp"][56])] + checks = { + "all_60_records_and_audited_values_finite": not failures, + "bp_mean_accuracy_at_least_0p90_each_depth": all( + statistics.mean(accuracies["bp"][depth]) >= 0.90 + for depth in DEPTHS), + "every_dynamic_accuracy_at_least_0p87": min( + value for depth in DEPTHS + for value in accuracies["dynamic"][depth]) >= 0.87, + "dynamic_mean_within_2pt_bp_each_depth": all( + statistics.mean(bp_deficits[depth]) <= 0.02 for depth in DEPTHS), + "dynamic_bp_deficit_upper_bound_at_most_3pt_each_depth": all( + upper_confidence_bound(bp_deficits[depth]) <= 0.03 + for depth in DEPTHS), + "dynamic_mean_within_1p5pt_kp_each_depth": all( + statistics.mean(kp_deficits[depth]) <= 0.015 for depth in DEPTHS), + "dynamic_kp_deficit_upper_bound_at_most_2p5pt_each_depth": all( + upper_confidence_bound(kp_deficits[depth]) <= 0.025 + for depth in DEPTHS), + "d56_dynamic_mean_advantage_over_dfa_at_least_2pt": + statistics.mean(d56_dfa_advantage) >= 0.02, + "d56_dynamic_dfa_advantage_lower_bound_at_least_1pt": + lower_confidence_bound(d56_dfa_advantage) >= 0.01, + "dynamic_mean_d20_to_d56_gain_at_least_0p5pt": + statistics.mean(dynamic_depth_gain) >= 0.005, + "dynamic_depth_gain_lower_bound_nonnegative": + lower_confidence_bound(dynamic_depth_gain) >= 0.0, + "at_least_four_dynamic_seeds_improve_with_depth": + sum(value > 0 for value in dynamic_depth_gain) >= 4, + "dynamic_depth_gain_within_1pt_of_bp_gain": + statistics.mean(dynamic_depth_gain) + >= statistics.mean(bp_depth_gain) - 0.01, + "dynamic_mean_alignment_at_least_0p85_each_depth": all( + statistics.mean(alignments[depth]) >= 0.85 for depth in DEPTHS), + "every_d56_dynamic_alignment_at_least_0p80": + min(alignments[56]) >= 0.80, + "d56_mean_alignment_retains_90pct_of_d20": + statistics.mean(alignments[56]) + >= 0.90 * statistics.mean(alignments[20]), + "all_mechanism_query_cost_memory_invariants": not failures, + } + passed = all(checks.values()) + output = { + "protocol": "oral_a_dynamic_innovation_scaling_v1", + "status": "passed" if passed else "failed", + "complete_grid": True, + "checks": checks, + "metrics": { + "accuracy_by_method_depth_seed": accuracies, + "mean_accuracy": {method: { + str(depth): statistics.mean(values) + for depth, values in by_depth.items()} + for method, by_depth in accuracies.items()}, + "bp_deficit": bp_deficits, + "kp_deficit": kp_deficits, + "d56_dfa_advantage": d56_dfa_advantage, + "dynamic_d20_to_d56_gain": dynamic_depth_gain, + "bp_d20_to_d56_gain": bp_depth_gain, + "dynamic_alignment": alignments, + "mechanism": mechanism, + "invariant_failures": failures, + }, + "d4_source_commit": next(iter(d4_commits)), + "new_source_commit": next(iter(new_commits)), + "source_sha256": source_sha256, + "protocol_sha256": sha256(PROTOCOL_PATH), + "runner_sha256": sha256(RUNNER_PATH), + "d4_gate_sha256": d4_digest, + "r2_gate_sha256": sha256(args.r2_gate), + "test_evaluations": 60, + "standard_depth_scaling_established": passed, + "review_score_before": 8, + "review_score_after": 9 if passed else 8, + "score_change_rule": ( + "only the complete R2-gated 60-cell standard-ResNet panel can " + "establish oral-A dynamic-innovation scaling"), + } + os.makedirs(os.path.dirname(os.path.abspath(args.out)), exist_ok=True) + if os.path.exists(args.out): + with open(args.out) as handle: + existing = json.load(handle) + require(existing == output, + "existing oral-A gate differs from deterministic re-audit") + print(json.dumps(output, indent=2)) + return + with open(args.out, "w") as handle: + json.dump(output, handle, indent=2, sort_keys=True) + handle.write("\n") + print(json.dumps(output, indent=2)) + + +if __name__ == "__main__": + main() diff --git a/experiments/finalize_accept.sh b/experiments/finalize_accept.sh index e13df52..95f95e9 100755 --- a/experiments/finalize_accept.sh +++ b/experiments/finalize_accept.sh @@ -16,7 +16,9 @@ experiments/finalize_claims.sh /home/yurenh2/miniconda3/envs/ep_pascal/bin/python3 -m py_compile \ experiments/bci_td_run.py experiments/analyze_bci_td_development.py \ experiments/bci_td_confirmation.py \ - experiments/analyze_bci_td_confirmation.py + experiments/analyze_bci_td_confirmation.py \ + experiments/oral_a_dynamic_scaling.py \ + experiments/analyze_oral_a_dynamic_scaling.py /home/yurenh2/miniconda3/envs/ep_pascal/bin/python3 \ experiments/analyze_kp_dynamic_projection.py >/dev/null /home/yurenh2/miniconda3/envs/ep_pascal/bin/python3 \ @@ -35,6 +37,10 @@ if [ -f results/bci_td_confirmation_gate.json ]; then /home/yurenh2/miniconda3/envs/ep_pascal/bin/python3 \ experiments/analyze_bci_td_confirmation.py >/dev/null fi +if [ -f results/oral_a_dynamic_scaling_gate.json ]; then + /home/yurenh2/miniconda3/envs/ep_pascal/bin/python3 \ + experiments/analyze_oral_a_dynamic_scaling.py >/dev/null +fi /home/yurenh2/miniconda3/bin/python \ experiments/plot_tracking_failure.py >/dev/null /home/yurenh2/miniconda3/bin/python \ diff --git a/experiments/oral_a_dynamic_scaling.py b/experiments/oral_a_dynamic_scaling.py new file mode 100755 index 0000000..3de4561 --- /dev/null +++ b/experiments/oral_a_dynamic_scaling.py @@ -0,0 +1,187 @@ +#!/usr/bin/env python3 +"""Run shards of the R2-gated standard-depth dynamic-innovation panel.""" +import argparse +import hashlib +import json +import os +import subprocess +import sys + + +ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +PROTOCOL_PATH = os.path.join(ROOT, "ORAL_A_RECOVERY.md") +DEPTHS = (20, 32, 56) +SEEDS = tuple(range(10, 15)) +METHODS = ("bp", "dfa", "clean_kp", "dynamic") + + +def sha256(path): + digest = hashlib.sha256() + with open(path, "rb") as handle: + for chunk in iter(lambda: handle.read(1024 * 1024), b""): + digest.update(chunk) + return digest.hexdigest() + + +def git_output(*args): + return subprocess.run( + ["git", *args], cwd=ROOT, check=True, capture_output=True, + text=True).stdout.strip() + + +def require_prerequisites(d4_path, r2_path): + with open(d4_path) as handle: + d4 = json.load(handle) + with open(r2_path) as handle: + r2 = json.load(handle) + d4_digest = sha256(d4_path) + if not (d4.get("protocol") == + "kp_dynamic_neutral_projection_confirmation_v1" + and d4.get("status") == "passed" + and d4.get("review_score_after") == 7): + raise ValueError("oral-A recovery requires the complete D4 pass") + if not (r2.get("protocol") == "oral_b_td_confirmation_v1" + and r2.get("status") == "passed" + and r2.get("complete_grid") is True + and r2.get("oral_b_plasticity_innovation_established") is True + and r2.get("review_score_after") == 8 + and r2.get("d4_gate_sha256") == d4_digest): + raise ValueError("oral-A recovery requires the complete oral-B R2 pass") + + paths = [ + os.path.abspath(__file__), PROTOCOL_PATH, + os.path.abspath(d4_path), os.path.abspath(r2_path), + os.path.join(ROOT, "experiments", "conv_run.py"), + ] + relative = [os.path.relpath(path, ROOT) for path in paths] + tracked = all(subprocess.run( + ["git", "ls-files", "--error-unmatch", path], cwd=ROOT, + capture_output=True).returncode == 0 for path in relative) + dirty = bool(git_output("status", "--porcelain", "--untracked-files=no")) + if dirty or not tracked: + raise RuntimeError( + "oral-A recovery requires clean tracked runner, protocol, and gates") + return { + "git_commit": git_output("rev-parse", "HEAD"), + "protocol_sha256": sha256(PROTOCOL_PATH), + "d4_gate_sha256": d4_digest, + "r2_gate_sha256": sha256(r2_path), + } + + +def job_command(method, depth, seed, device, outdir): + if method in ("clean_kp", "dynamic") and depth == 20: + raise ValueError("depth-20 clean-KP/dynamic cells are reused from D4") + common = [ + sys.executable, "experiments/conv_run.py", + "--device", device, "--depth", str(depth), "--width", "16", + "--seed", str(seed), "--loader_seed", str(seed), + "--batch_size", "128", "--epochs", "200", "--train_limit", "0", + "--val_examples", "0", "--split_seed", "2027", + "--eval_split", "test", "--eval_every", "0", "--augment_train", "1", + "--lr_schedule", "step", "--lr_milestones", "100,150", + "--lr_gamma", "0.1", "--warmup_epochs", "0", + "--momentum", "0.9", "--weight_decay", "1e-4", + "--normalization", "batchnorm", "--a_scale", "1", + ] + if method == "bp": + specific = ["--mode", "bp", "--lr", "0.1"] + elif method == "dfa": + specific = [ + "--mode", "dfa", "--lr", "0.03", "--output_lr", "0.1", + "--vectorizer_mode", "spatial_template", "--alignment_probe", "32", + ] + elif method == "clean_kp": + specific = [ + "--mode", "kp", "--lr", "0.1", "--output_lr", "0.1", + "--alignment_probe", "32", + ] + elif method == "dynamic": + specific = [ + "--mode", "kp_traffic", "--lr", "0.1", "--output_lr", "0.1", + "--traffic_rule", "innovation", "--predictor_mode", "closed_form", + "--neutral_projection", "1", "--traffic_seed", str(5000 + seed), + "--traffic_ratio", "4", "--traffic_calibration_examples", "64", + "--learn_P", "1", "--eta_P", "0.1", + "--predictor_warmup_steps", "1", "--predictor_every", "0", + "--alignment_probe", "32", + ] + else: + raise ValueError(f"unknown oral-A method: {method}") + path = os.path.join(outdir, f"{method}_d{depth}_s{seed}.json") + return path, common + specific + ["--out", path] + + +def jobs(device, outdir): + result = [] + for depth in DEPTHS: + for seed in SEEDS: + for method in METHODS: + if depth == 20 and method in ("clean_kp", "dynamic"): + continue + path, command = job_command(method, depth, seed, device, outdir) + result.append((method, depth, seed, path, command)) + return result + + +def existing_matches(path, method, depth, seed, source_commit): + with open(path) as handle: + row = json.load(handle) + expected_mode = { + "bp": "bp", "dfa": "dfa", "clean_kp": "kp", "dynamic": "kp_traffic", + }[method] + args = row.get("args", {}) + return (args.get("mode") == expected_mode + and args.get("depth") == depth + and args.get("seed") == seed + and row.get("provenance", {}).get("git_commit") == source_commit + and row.get("provenance", {}).get("git_tracked_dirty") is False) + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument( + "--d4_gate", + default="results/kp_dynamic_projection_confirmation_gate.json") + parser.add_argument( + "--r2_gate", default="results/bci_td_confirmation_gate.json") + parser.add_argument("--device", default="cuda") + parser.add_argument("--method", choices=("all",) + METHODS, default="all") + parser.add_argument("--depth", type=int, choices=DEPTHS) + parser.add_argument("--seed", type=int, choices=SEEDS) + parser.add_argument("--shard-index", type=int, default=0) + parser.add_argument("--num-shards", type=int, default=1) + parser.add_argument("--outdir", default="results/oral_a_dynamic_scaling") + parser.add_argument("--dry-run", action="store_true") + args = parser.parse_args() + if not 0 <= args.shard_index < args.num_shards: + raise ValueError("invalid shard index") + source = require_prerequisites(args.d4_gate, args.r2_gate) + selected = jobs(args.device, args.outdir) + if args.method != "all": + selected = [job for job in selected if job[0] == args.method] + if args.depth is not None: + selected = [job for job in selected if job[1] == args.depth] + if args.seed is not None: + selected = [job for job in selected if job[2] == args.seed] + selected = [job for index, job in enumerate(selected) + if index % args.num_shards == args.shard_index] + if not selected: + raise ValueError("no oral-A recovery jobs match the requested shard") + + os.makedirs(args.outdir, exist_ok=True) + for method, depth, seed, path, command in selected: + tag = f"{method}_d{depth}_s{seed}" + if os.path.exists(path): + if not existing_matches( + path, method, depth, seed, source["git_commit"]): + raise RuntimeError(f"refusing mismatched existing record: {path}") + print(f"preserving {tag}", flush=True) + continue + print(tag, " ".join(command), flush=True) + if not args.dry_run: + subprocess.run(command, cwd=ROOT, check=True) + + +if __name__ == "__main__": + main() |
