diff options
| -rw-r--r-- | ORAL_A_V2.md | 22 | ||||
| -rw-r--r-- | README.md | 5 | ||||
| -rw-r--r-- | RESULTS.md | 33 | ||||
| -rw-r--r-- | REVIEW_SCORECARD.md | 2 | ||||
| -rw-r--r-- | ROADMAP.md | 9 | ||||
| -rw-r--r-- | experiments/analyze_oral_a_failure.py | 44 | ||||
| -rw-r--r-- | results/oral_a_failure_diagnosis.json | 23 | ||||
| -rw-r--r-- | results/oral_a_v2_calibration/channel_subspace_etaA0.01.json | 393 | ||||
| -rw-r--r-- | results/oral_a_v2_calibration/channel_subspace_etaA0.1.json | 393 | ||||
| -rw-r--r-- | results/oral_a_v2_calibration/channel_subspace_etaA1.0.json | 393 | ||||
| -rw-r--r-- | results/oral_a_v2_calibration/unit_targets_etaA0.01.json | 390 | ||||
| -rw-r--r-- | results/oral_a_v2_calibration/unit_targets_etaA0.1.json | 390 | ||||
| -rw-r--r-- | results/oral_a_v2_calibration/unit_targets_etaA1.0.json | 390 | ||||
| -rw-r--r-- | results/oral_a_v2_calibration_gate.json | 147 |
14 files changed, 2612 insertions, 22 deletions
diff --git a/ORAL_A_V2.md b/ORAL_A_V2.md index d4586f6..f77c5d2 100644 --- a/ORAL_A_V2.md +++ b/ORAL_A_V2.md @@ -103,3 +103,25 @@ mechanics and causal capture rather than standard-scale task success. A V2-2 pass can move the score from 5 only after its full audit is committed. A V2-3 multi-depth, multi-seed pass is required for an oral-level scaling claim. +## Audited outcome (2026-07-22) + +All six V2-1 records were finite and shared clean source commit `fc8fe99`. +The frozen selector chose `eta_A=0.01` for both estimators. Structured +calibration improved exact teaching/negative-gradient alignment: + +| estimator | early-third alignment | all-layer alignment | +|:--|--:|--:| +| unit targets | 0.001105 | 0.011664 | +| channel subspace | 0.007209 | 0.052740 | + +This is a real 6.5x early-layer and 4.5x all-layer improvement at identical +causal-query count, but it fails two frozen advancement checks: early-third +alignment is below `0.01`, and its absolute gain over unit targets is `0.006104` +rather than `0.01`. The all-layer check passes because the late blocks reach +substantially higher alignment; the earliest blocks remain the bottleneck. +The recorded target powers (`306.807` for the full hidden field and `0.270893` +for channel-basis moments) are intentionally not divided or compared: the two +estimators report different metric spaces. + +V2-1 status is **failed**. V2-2 was not launched, no test endpoint or +confirmation seed was touched, and the strict reviewer score remains 5/10. @@ -45,6 +45,11 @@ and scaling behavior. See `NOVELTY.md` for the exact prior-art boundary. became nonfinite at epoch 89 and ended at chance, while DFA remained finite at `33.06%`. A4 was therefore not opened and no confirmation test seed was touched. +- The post-failure representable-subspace estimator passed its exact mechanics + tests and improved frozen-forward early/all-layer alignment from + `0.0011/0.0117` to `0.0072/0.0527` at matched query count. It nevertheless + missed the frozen `0.01` early-layer and improvement gates, so no v2 full + ResNet run was launched. - Native author-code fidelity is complete. BurstCCN reaches `80.10%` at its validation-selected epoch versus published `82.97 +/- 0.21%`; Dual Prop reaches `92.46%` versus published `92.41 +/- 0.07%`. Their audited walls are @@ -635,12 +635,33 @@ The executable post-failure diagnosis finds a specific precursor rather than only a bad endpoint. Before nonfiniteness, the apical prediction--target cosine never exceeds `8.51e-5` in magnitude, while calibration MSE differs from target power by at most `6.96e-8` relatively. Target power grows `2560.7x` from epoch -1 to 80 and `2.13e9x` by epoch 87. Thus the positive short-run teaching -alignment cannot be attributed to successful causal calibration of `A`; the -full-spatial K1 estimator is effectively all noise after projection into the -small channel-gated vectorizer. This is a post-failure diagnosis, not a passed -preregistered result, and motivates a separately labelled representable- -subspace estimator rather than a threshold change to Oral-A v1. +1 to 80 and `2.13e9x` by epoch 87. These instantaneous metrics show that +stochastic unit targets dominate the regression signal, but the later v2 +experiment also establishes an important interpretation limit: near-zero +single-event target cosine does not by itself prove zero conditional learning. + +The separately frozen post-failure v2 screen perturbs only the two basis fields +expressible by the channel-gated vectorizer. At identical query count and 400 +frozen-forward calibration minibatches, selected `eta_A=0.01` gives: + +| estimator | calibration metric space | early-third alignment | all-layer alignment | +|:--|:--|--:|--:| +| unit targets | full hidden field | 0.001105 | 0.011664 | +| channel subspace | channel-basis moments | 0.007209 | 0.052740 | + +Thus representable-subspace perturbation improves early/all-layer +exact-gradient alignment by 6.5x/4.5x at matched causal-query count. +However, it misses the frozen early alignment threshold (`0.007209 < 0.01`) +and the required absolute advantage (`0.006104 < 0.01`). V2-1 therefore fails, +the full v2 ResNet run is not launched, and confirmation remains untouched. +The useful conclusion is narrower: spatial projection was a real variance +problem, but removing it does not yet deliver sufficient early-layer causal +credit at standard depth. + +The two target-power logs are not directly comparable: unit targets are scored +per full hidden field, whereas structured targets are scored per channel-basis +moment. No empirical 1133x variance-reduction claim is made from their raw +ratio. ## How to run `experiments/run.py --mode {bp,fa,dfa,sdil} --dataset {mnist,fmnist,cifar10} --depth D --residual {0,1} --act {tanh,gelu,silu,relu}` diff --git a/REVIEW_SCORECARD.md b/REVIEW_SCORECARD.md index df15681..846cb22 100644 --- a/REVIEW_SCORECARD.md +++ b/REVIEW_SCORECARD.md @@ -58,6 +58,7 @@ Every formal result report records: | Native baselines complete | 5 | BurstCCN is below its published endpoint; Dual Prop reproduces 92.46% versus 92.41%, with strict provenance and cost semantics | Fairness objection narrows, but SDIL gains no standard-scale evidence | | Oral-A A1/A2 | 5 | BP reached 91.62%; short channel-gated SDIL reached 41.98% versus tuned DFA at 37.16% | Development screening alone cannot raise the score | | Oral-A A3 fails | 5 | Full ResNet-20 SDIL became nonfinite at epoch 89 and ended at 10%; DFA ended finite at 33.06% | Standard-scale and oral-A claims are closed; A4 remains untouched | +| Oral-A-v2 causal capture fails | 5 | Structured perturbation improves early/all-layer alignment 6.5x/4.5x but misses the frozen early gate | Mechanism diagnosis sharpens; no full standard-scale endpoint opens | | Oral-A A4 | not opened | The prerequisite A3 gate failed | No oral-A confirmation claim is available | These are conditional reviewer forecasts, not promised scores. A failed stage leaves its negative @@ -72,6 +73,7 @@ retroactively reopened by success on standard vision benchmarks. | 2026-07-22 / `c753f51`, `1b24c87`, `6d19078` | Existing 60-run innovation panel promoted to a strict main figure; conditional-projection and norm-direction identities made executable | 5 → 5 | Closes a presentation/theory objection and makes the narrow novelty legible, but adds no new held-out evidence and therefore earns no score inflation | | 2026-07-22 / frozen Oral-A A1--A3 | A1 and A2 pass; full A3 SDIL becomes nonfinite and fails four of six checks; A4 untouched | 5 → 5 | Closes the standard-scale question negatively. The narrow mechanism paper survives, while any standard-ResNet or oral claim does not | | 2026-07-22 / native C4 | BurstCCN and Dual Prop author-code records pass the strict audit; Dual Prop reproduces 92.46% test in 23119.8 s | 5 → 5 | Closes a baseline-fidelity objection and confirms a strong expensive comparator, but does not repair SDIL's failed A3 evidence | +| 2026-07-22 / Oral-A-v2 V2-1 | Six clean frozen-forward records: structured calibration raises early/all-layer alignment to 0.0072/0.0527 but fails two frozen advancement checks | 5 → 5 | Confirms representable-subspace variance was real, while showing that early-layer causal credit remains below the standard-depth gate; V2-2 and confirmation stay closed | Future rows are appended only after an audited frozen stage. A score staying flat is informative: engineering, theory exposition, or visualization may make the paper more defensible without @@ -355,6 +355,15 @@ development branch must therefore reduce causal-estimator variance in the representable channel-gated subspace; changing only the v1 threshold is not an allowed response. +**Post-failure v2 status: causal-capture gate failed.** The structured +representable-subspace estimator raised matched frozen-forward early/all-layer alignment from +`0.001105/0.011664` to `0.007209/0.052740`. It passed the all-layer threshold +but missed both the preregistered `0.01` early-layer threshold and the `0.01` +absolute early-layer advantage. Per `ORAL_A_V2.md`, the full 200-epoch v2 run +was not launched; test and confirmation seeds remain untouched. This localizes +the remaining bottleneck to early-layer credit rather than merely spatial +projection variance. + Prepare convolutional local-update primitives and ResNet-20/32/56 protocols early. Queue frozen runs opportunistically on authorized idle GPUs. Because BurstCCN already reports CIFAR-10 and ImageNet scaling, dataset scale alone is not novel. The oral-level target is a memorable joint diff --git a/experiments/analyze_oral_a_failure.py b/experiments/analyze_oral_a_failure.py index 9f84bbf..bf68d12 100644 --- a/experiments/analyze_oral_a_failure.py +++ b/experiments/analyze_oral_a_failure.py @@ -24,12 +24,15 @@ def main(): parser.add_argument( "--gate", default="results/oral_a_full_gate.json") parser.add_argument( + "--v2_gate", default="results/oral_a_v2_calibration_gate.json") + parser.add_argument( "--out", default="results/oral_a_failure_diagnosis.json") args = parser.parse_args() full = load(args.full) short = load(args.short) gate = load(args.gate) + v2_gate = load(args.v2_gate) expected = { "mode": "sdil", "depth": 20, "epochs": 200, "vectorizer_mode": "channel_gated", "a_scale": 1.0, @@ -42,6 +45,9 @@ def main(): raise ValueError("full A3 result has tracked source edits") if gate["status"] != "failed" or gate["confirmation_test_seeds_touched"]: raise ValueError("expected a failed A3 gate with untouched confirmation") + if (v2_gate["status"] != "failed" + or v2_gate["confirmation_test_seeds_touched"]): + raise ValueError("expected a failed v2 gate with untouched confirmation") calibration_rows = [] first_nonfinite_epoch = None @@ -82,9 +88,14 @@ def main(): max_mse_power_gap = max(abs(row["mse_to_target_power"] - 1.0) for row in calibration_rows) short_calibration = short["apical_warmup"]["last"] + unit = v2_gate["selected"]["unit_targets"] + structured = v2_gate["selected"]["channel_subspace"] output = { - "protocol": "oral_a_A3_failure_diagnosis_v1", - "source_paths": {"full": args.full, "short": args.short, "gate": args.gate}, + "protocol": "oral_a_A3_failure_diagnosis_v2", + "source_paths": { + "full": args.full, "short": args.short, "gate": args.gate, + "v2_gate": args.v2_gate, + }, "source_commits": { "full": full["provenance"]["git_commit"], "short": short["provenance"]["git_commit"], @@ -113,17 +124,32 @@ def main(): "max_abs_prediction_target_cosine_before_nonfinite": max_abs_cosine, "max_abs_mse_to_target_power_minus_one": max_mse_power_gap, }, + "post_failure_v2_refinement": { + "unit_target_early_third_alignment": ( + unit["metrics"]["early_third_alignment"]), + "unit_target_all_layer_alignment": ( + unit["metrics"]["all_layer_alignment"]), + "structured_early_third_alignment": ( + structured["metrics"]["early_third_alignment"]), + "structured_all_layer_alignment": ( + structured["metrics"]["all_layer_alignment"]), + "structured_gate_status": v2_gate["status"], + }, "diagnosis": { - "classification": "causal_target_not_captured_before_runaway", + "classification": "unit_target_regression_snr_inadequate_before_runaway", "evidence": [ - "prediction-target cosine remains effectively zero", - "calibration MSE remains indistinguishable from target power", + "instantaneous prediction-target cosine remains effectively zero", + "calibration MSE remains indistinguishable from stochastic target power", "target power grows by orders of magnitude before nonfiniteness", - "short-run teaching alignment therefore cannot be attributed to learned A", + "matched frozen-forward unit targets later yield only 0.0011 early-layer alignment", ], - "next_test": ( - "reduce estimator variance by perturbing the representable " - "channel-gated feedback subspace, not every spatial unit"), + "interpretation_limit": ( + "instantaneous target metrics alone do not prove zero conditional " + "learning; structured v2 has low instantaneous cosine yet positive " + "exact-gradient alignment"), + "next_test_outcome": ( + "representable-subspace perturbation improved exact alignment but " + "failed the frozen early-layer causal-capture gate"), }, } os.makedirs(os.path.dirname(os.path.abspath(args.out)), exist_ok=True) diff --git a/results/oral_a_failure_diagnosis.json b/results/oral_a_failure_diagnosis.json index 7c6b31b..5239e8b 100644 --- a/results/oral_a_failure_diagnosis.json +++ b/results/oral_a_failure_diagnosis.json @@ -44,14 +44,15 @@ "target_power_growth_epoch1_to_87": 2126827049.3932126 }, "diagnosis": { - "classification": "causal_target_not_captured_before_runaway", + "classification": "unit_target_regression_snr_inadequate_before_runaway", "evidence": [ - "prediction-target cosine remains effectively zero", - "calibration MSE remains indistinguishable from target power", + "instantaneous prediction-target cosine remains effectively zero", + "calibration MSE remains indistinguishable from stochastic target power", "target power grows by orders of magnitude before nonfiniteness", - "short-run teaching alignment therefore cannot be attributed to learned A" + "matched frozen-forward unit targets later yield only 0.0011 early-layer alignment" ], - "next_test": "reduce estimator variance by perturbing the representable channel-gated feedback subspace, not every spatial unit" + "interpretation_limit": "instantaneous target metrics alone do not prove zero conditional learning; structured v2 has low instantaneous cosine yet positive exact-gradient alignment", + "next_test_outcome": "representable-subspace perturbation improved exact alignment but failed the frozen early-layer causal-capture gate" }, "frozen_outcome": { "confirmation_test_seeds_touched": false, @@ -71,7 +72,14 @@ "loss": null } }, - "protocol": "oral_a_A3_failure_diagnosis_v1", + "post_failure_v2_refinement": { + "structured_all_layer_alignment": 0.05273995646520665, + "structured_early_third_alignment": 0.007209055746595065, + "structured_gate_status": "failed", + "unit_target_all_layer_alignment": 0.011664319529173602, + "unit_target_early_third_alignment": 0.0011051012067279469 + }, + "protocol": "oral_a_A3_failure_diagnosis_v2", "short_screen_contrast": { "accuracy": 0.4198, "early_third_alignment": 0.08889565182228883, @@ -85,6 +93,7 @@ "source_paths": { "full": "results/oral_a_dev/sdil_full_r20_s0.json", "gate": "results/oral_a_full_gate.json", - "short": "results/oral_a_short/sdil_channel_gated_lr0.03.json" + "short": "results/oral_a_short/sdil_channel_gated_lr0.03.json", + "v2_gate": "results/oral_a_v2_calibration_gate.json" } } diff --git a/results/oral_a_v2_calibration/channel_subspace_etaA0.01.json b/results/oral_a_v2_calibration/channel_subspace_etaA0.01.json new file mode 100644 index 0000000..4282d51 --- /dev/null +++ b/results/oral_a_v2_calibration/channel_subspace_etaA0.01.json @@ -0,0 +1,393 @@ +{ + "apical_warmup": { + "first": { + "calibration_mse": 0.667037273962917, + "parameter_update_rms": 0.023085309606734133, + "prediction_target_cosine": 0.0, + "target_power": 0.667037273962917 + }, + "last": { + "calibration_mse": 0.014343140418299469, + "parameter_update_rms": 0.00343628282575056, + "prediction_target_cosine": -0.005531953824674762, + "target_power": 0.014335632392748118 + }, + "mean": { + "calibration_mse": 0.2708950854477958, + "parameter_update_rms": 0.011981565658686305, + "prediction_target_cosine": 0.00018159585958168355, + "target_power": 0.2708926473589623 + }, + "steps": 400 + }, + "architecture": { + "adaptive_apical_parameters": 390592, + "base_width": 16, + "blocks_per_stage": 3, + "bn_eps": 1e-05, + "bn_momentum": 0.1, + "depth": 20, + "family": "CIFAR 6n+2 ResNet, option-A shortcuts", + "fixed_traffic_coefficients": 188416, + "forward_parameters": 269722, + "hidden_shapes": [ + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ] + ], + "normalization": "batchnorm", + "predictor_parameters": 376832, + "residual_scale": 1.0, + "vectorizer_mode": "channel_gated", + "vectorizer_parameters": 13760 + }, + "args": { + "a_scale": 0.0, + "a_warmup_steps": 400, + "alignment_probe": 64, + "apical_calibration_mode": "channel_subspace", + "apical_seed": null, + "augment_train": 1, + "batch_size": 128, + "bn_eps": 1e-05, + "bn_momentum": 0.1, + "data_dir": "/home/yurenh2/sdrn/data", + "depth": 20, + "device": "cuda:0", + "epochs": 0, + "eta_A": 0.01, + "eta_P": 0.01, + "eval_every": 0, + "eval_split": "validation", + "learn_P": 0, + "loader_seed": 0, + "lr": 0.03, + "lr_gamma": 0.1, + "lr_milestones": "100,150", + "lr_schedule": "constant", + "max_steps": 0, + "mode": "sdil", + "momentum": 0.9, + "normalization": "batchnorm", + "nuisance_scale": 0.0, + "out": "results/oral_a_v2_calibration/channel_subspace_etaA0.01.json", + "output_lr": 0.1, + "pert_directions": 1, + "pert_every": 4, + "pert_sigma": 0.01, + "perturb_seed": 1000, + "predictor_warmup_steps": 0, + "residual_scale": null, + "seed": 0, + "split_seed": 2027, + "train_limit": 10000, + "use_residual": 1, + "val_examples": 5000, + "vectorizer_mode": "channel_gated", + "warmup_epochs": 0, + "weight_decay": 0.0001, + "weight_scale": 1.0, + "width": 16 + }, + "calibration_metric_space": "channel_basis_moments", + "counters": { + "apical_warmup_examples": 50640, + "calibration_event_examples": 50640, + "causal_scalar_observations": 800, + "logical_batch_loss_queries": 800, + "ordinary_examples": 0, + "per_example_loss_terms": 101280, + "perturbation_events": 400, + "perturbation_forward_examples": 101280, + "predictor_warmup_examples": 0 + }, + "diagnostics": { + "early_third_mean": 0.007209055746595065, + "innovation_negative_gradient_cosine": [ + 0.0021500173024833202, + 0.005721019580960274, + 0.008716392330825329, + 0.01114893238991499, + 0.005342159885913134, + 0.010175812989473343, + 0.006470849737524986, + 0.013490993529558182, + 0.025294749066233635, + 0.028453536331653595, + 0.041042692959308624, + 0.018679656088352203, + 0.04189874976873398, + 0.04070262610912323, + 0.09365405142307281, + 0.07153154909610748, + 0.11656917631626129, + 0.19568055868148804, + 0.26533564925193787 + ], + "normalization_state": "training_batch_stats_without_running_update", + "raw_negative_gradient_cosine": [ + 0.0021500173024833202, + 0.005721019580960274, + 0.008716392330825329, + 0.01114893238991499, + 0.005342159885913134, + 0.010175812989473343, + 0.006470849737524986, + 0.013490993529558182, + 0.025294749066233635, + 0.028453536331653595, + 0.041042692959308624, + 0.018679656088352203, + 0.04189874976873398, + 0.04070262610912323, + 0.09365405142307281, + 0.07153154909610748, + 0.11656917631626129, + 0.19568055868148804, + 0.26533564925193787 + ], + "teaching_negative_gradient_cosine": [ + 0.0021500173024833202, + 0.005721019580960274, + 0.008716392330825329, + 0.01114893238991499, + 0.005342159885913134, + 0.010175812989473343, + 0.006470849737524986, + 0.013490993529558182, + 0.025294749066233635, + 0.028453536331653595, + 0.041042692959308624, + 0.018679656088352203, + 0.04189874976873398, + 0.04070262610912323, + 0.09365405142307281, + 0.07153154909610748, + 0.11656917631626129, + 0.19568055868148804, + 0.26533564925193787 + ], + "wall_s": 0.11891794204711914 + }, + "epochs": [], + "evaluation_protocol": { + "test_evaluations": 0, + "test_used_for_selection": false, + "validation_evaluations": 1 + }, + "final": { + "accuracy": 0.1004, + "epoch": 0, + "evaluation_split": "validation", + "finite": true, + "loss": 16.593759375, + "step": 0 + }, + "hardware": { + "cuda_device_name": "NVIDIA GeForce GTX 1080", + "cuda_visible_devices": "7", + "device": "cuda:0", + "device_total_memory_bytes": 8507949056, + "peak_memory_allocated_bytes": 1734868480, + "peak_memory_reserved_bytes": 2575302656, + "torch_version": "2.3.1+cu118" + }, + "protocol_family": "oral_a_cifar_local_resnet_development", + "provenance": { + "git_commit": "fc8fe99504fe86a3636721031a7dc3d41a7909a6", + "git_tracked_dirty": false + }, + "schema_version": 1, + "split": { + "cifar_source_files": [ + { + "bytes": 31035704, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_1", + "sha256": "54636561a3ce25bd3e19253c6b0d8538147b0ae398331ac4a2d86c6d987368cd" + }, + { + "bytes": 31035320, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_2", + "sha256": "766b2cef9fbc745cf056b3152224f7cf77163b330ea9a15f9392beb8b89bc5a8" + }, + { + "bytes": 31035999, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_3", + "sha256": "0f00d98ebfb30b3ec0ad19f9756dc2630b89003e10525f5e148445e82aa6a1f9" + }, + { + "bytes": 31035696, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_4", + "sha256": "3f7bb240661948b8f4d53e36ec720d8306f5668bd0071dcb4e6c947f78e9682b" + }, + { + "bytes": 31035623, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_5", + "sha256": "d91802434d8376bbaeeadf58a737e3a1b12ac839077e931237e0dcd43adcb154" + }, + { + "bytes": 31035526, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/test_batch", + "sha256": "f53d8d457504f7cff4ea9e021afcf0e0ad8e24a91f3fc42091b8adef61157831" + } + ], + "dataset": "cifar10", + "evaluation_split": "validation", + "input_layout": "NCHW", + "input_shape": [ + 3, + 32, + 32 + ], + "loader_seed": 0, + "normalization_mean": [ + 0.49140000343322754, + 0.4821999967098236, + 0.4465000033378601 + ], + "normalization_std": [ + 0.24699999392032623, + 0.2434999942779541, + 0.26159998774528503 + ], + "split_from_training_only": true, + "split_seed": 2027, + "test_examples": 10000, + "train_examples": 10000, + "training_augmentation": "random_crop_32_padding_4_zero_then_horizontal_flip_p0.5", + "validation_class_counts": { + "0": 500, + "1": 500, + "2": 500, + "3": 500, + "4": 500, + "5": 500, + "6": 500, + "7": 500, + "8": 500, + "9": 500 + }, + "validation_examples": 5000, + "validation_index_sha256": "8328b206a97c420e49e54e3eca4abe3274c4756b084355784ea3fb8059e4515b" + }, + "timing": { + "apical_warmup_wall_s": 23.970787286758423, + "evaluation_wall_s": 0.3708672523498535, + "predictor_warmup_wall_s": 0.0, + "timing_excludes_data_loading_hashing_and_model_construction": true, + "total_timed_wall_s": 24.341848134994507, + "train_wall_s": 0.0 + }, + "work": { + "apical_macs_per_example": 202176, + "causal_scalar_observations": 800, + "components": { + "apical_projection_macs": 10238192640, + "apical_regression_macs": 10238192640, + "bp_reverse_macs_estimate": 0, + "local_weight_correlation_macs": 0, + "ordinary_forward_macs": 0, + "perturbation_forward_macs": 4107009331200, + "warmup_clean_forward_macs": 2053504665600 + }, + "definition": "multiply-accumulates in conv/linear maps; one local weight correlation equals one forward-weight MAC count; BP reverse is estimated as one weight-gradient plus one activation-gradient convolution per forward convolution; elementwise nonlinearities and optimizer arithmetic excluded", + "forward_macs_per_example": 40551040, + "logical_batch_loss_queries": 800, + "per_example_cross_entropy_terms": 101280, + "total_clean_forward_examples": 50640, + "total_forward_equivalent_examples": 151920, + "total_macs_estimate": 6180990382080 + } +} diff --git a/results/oral_a_v2_calibration/channel_subspace_etaA0.1.json b/results/oral_a_v2_calibration/channel_subspace_etaA0.1.json new file mode 100644 index 0000000..95a7fbb --- /dev/null +++ b/results/oral_a_v2_calibration/channel_subspace_etaA0.1.json @@ -0,0 +1,393 @@ +{ + "apical_warmup": { + "first": { + "calibration_mse": 0.667037273962917, + "parameter_update_rms": 0.023085309606734133, + "prediction_target_cosine": 0.0, + "target_power": 0.667037273962917 + }, + "last": { + "calibration_mse": 0.014407349533643737, + "parameter_update_rms": 0.0035494557764320068, + "prediction_target_cosine": -0.0026674387418668616, + "target_power": 0.014335632392748118 + }, + "mean": { + "calibration_mse": 0.27095736647643465, + "parameter_update_rms": 0.012069313096218597, + "prediction_target_cosine": 0.00010544442092045836, + "target_power": 0.2708926473589623 + }, + "steps": 400 + }, + "architecture": { + "adaptive_apical_parameters": 390592, + "base_width": 16, + "blocks_per_stage": 3, + "bn_eps": 1e-05, + "bn_momentum": 0.1, + "depth": 20, + "family": "CIFAR 6n+2 ResNet, option-A shortcuts", + "fixed_traffic_coefficients": 188416, + "forward_parameters": 269722, + "hidden_shapes": [ + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ] + ], + "normalization": "batchnorm", + "predictor_parameters": 376832, + "residual_scale": 1.0, + "vectorizer_mode": "channel_gated", + "vectorizer_parameters": 13760 + }, + "args": { + "a_scale": 0.0, + "a_warmup_steps": 400, + "alignment_probe": 64, + "apical_calibration_mode": "channel_subspace", + "apical_seed": null, + "augment_train": 1, + "batch_size": 128, + "bn_eps": 1e-05, + "bn_momentum": 0.1, + "data_dir": "/home/yurenh2/sdrn/data", + "depth": 20, + "device": "cuda:0", + "epochs": 0, + "eta_A": 0.1, + "eta_P": 0.01, + "eval_every": 0, + "eval_split": "validation", + "learn_P": 0, + "loader_seed": 0, + "lr": 0.03, + "lr_gamma": 0.1, + "lr_milestones": "100,150", + "lr_schedule": "constant", + "max_steps": 0, + "mode": "sdil", + "momentum": 0.9, + "normalization": "batchnorm", + "nuisance_scale": 0.0, + "out": "results/oral_a_v2_calibration/channel_subspace_etaA0.1.json", + "output_lr": 0.1, + "pert_directions": 1, + "pert_every": 4, + "pert_sigma": 0.01, + "perturb_seed": 1000, + "predictor_warmup_steps": 0, + "residual_scale": null, + "seed": 0, + "split_seed": 2027, + "train_limit": 10000, + "use_residual": 1, + "val_examples": 5000, + "vectorizer_mode": "channel_gated", + "warmup_epochs": 0, + "weight_decay": 0.0001, + "weight_scale": 1.0, + "width": 16 + }, + "calibration_metric_space": "channel_basis_moments", + "counters": { + "apical_warmup_examples": 50640, + "calibration_event_examples": 50640, + "causal_scalar_observations": 800, + "logical_batch_loss_queries": 800, + "ordinary_examples": 0, + "per_example_loss_terms": 101280, + "perturbation_events": 400, + "perturbation_forward_examples": 101280, + "predictor_warmup_examples": 0 + }, + "diagnostics": { + "early_third_mean": 0.0038381816702894866, + "innovation_negative_gradient_cosine": [ + -0.0012592253042384982, + 0.00399992149323225, + 0.006424956023693085, + 0.005883498582988977, + 0.006901413202285767, + 0.0010785260237753391, + 0.002761237556114793, + 0.010423609055578709, + 0.009150509722530842, + 0.017207007855176926, + 0.015042684972286224, + 0.00847631599754095, + 0.012668993324041367, + 0.03519224375486374, + 0.042540062218904495, + 0.011548444628715515, + 0.04734557867050171, + 0.10991379618644714, + 0.15230399370193481 + ], + "normalization_state": "training_batch_stats_without_running_update", + "raw_negative_gradient_cosine": [ + -0.0012592253042384982, + 0.00399992149323225, + 0.006424956023693085, + 0.005883498582988977, + 0.006901413202285767, + 0.0010785260237753391, + 0.002761237556114793, + 0.010423609055578709, + 0.009150509722530842, + 0.017207007855176926, + 0.015042684972286224, + 0.00847631599754095, + 0.012668993324041367, + 0.03519224375486374, + 0.042540062218904495, + 0.011548444628715515, + 0.04734557867050171, + 0.10991379618644714, + 0.15230399370193481 + ], + "teaching_negative_gradient_cosine": [ + -0.0012592253042384982, + 0.00399992149323225, + 0.006424956023693085, + 0.005883498582988977, + 0.006901413202285767, + 0.0010785260237753391, + 0.002761237556114793, + 0.010423609055578709, + 0.009150509722530842, + 0.017207007855176926, + 0.015042684972286224, + 0.00847631599754095, + 0.012668993324041367, + 0.03519224375486374, + 0.042540062218904495, + 0.011548444628715515, + 0.04734557867050171, + 0.10991379618644714, + 0.15230399370193481 + ], + "wall_s": 0.11558938026428223 + }, + "epochs": [], + "evaluation_protocol": { + "test_evaluations": 0, + "test_used_for_selection": false, + "validation_evaluations": 1 + }, + "final": { + "accuracy": 0.1004, + "epoch": 0, + "evaluation_split": "validation", + "finite": true, + "loss": 16.593759375, + "step": 0 + }, + "hardware": { + "cuda_device_name": "NVIDIA GeForce GTX 1080", + "cuda_visible_devices": "5", + "device": "cuda:0", + "device_total_memory_bytes": 8507949056, + "peak_memory_allocated_bytes": 1734868480, + "peak_memory_reserved_bytes": 2575302656, + "torch_version": "2.3.1+cu118" + }, + "protocol_family": "oral_a_cifar_local_resnet_development", + "provenance": { + "git_commit": "fc8fe99504fe86a3636721031a7dc3d41a7909a6", + "git_tracked_dirty": false + }, + "schema_version": 1, + "split": { + "cifar_source_files": [ + { + "bytes": 31035704, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_1", + "sha256": "54636561a3ce25bd3e19253c6b0d8538147b0ae398331ac4a2d86c6d987368cd" + }, + { + "bytes": 31035320, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_2", + "sha256": "766b2cef9fbc745cf056b3152224f7cf77163b330ea9a15f9392beb8b89bc5a8" + }, + { + "bytes": 31035999, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_3", + "sha256": "0f00d98ebfb30b3ec0ad19f9756dc2630b89003e10525f5e148445e82aa6a1f9" + }, + { + "bytes": 31035696, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_4", + "sha256": "3f7bb240661948b8f4d53e36ec720d8306f5668bd0071dcb4e6c947f78e9682b" + }, + { + "bytes": 31035623, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_5", + "sha256": "d91802434d8376bbaeeadf58a737e3a1b12ac839077e931237e0dcd43adcb154" + }, + { + "bytes": 31035526, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/test_batch", + "sha256": "f53d8d457504f7cff4ea9e021afcf0e0ad8e24a91f3fc42091b8adef61157831" + } + ], + "dataset": "cifar10", + "evaluation_split": "validation", + "input_layout": "NCHW", + "input_shape": [ + 3, + 32, + 32 + ], + "loader_seed": 0, + "normalization_mean": [ + 0.49140000343322754, + 0.4821999967098236, + 0.4465000033378601 + ], + "normalization_std": [ + 0.24699999392032623, + 0.2434999942779541, + 0.26159998774528503 + ], + "split_from_training_only": true, + "split_seed": 2027, + "test_examples": 10000, + "train_examples": 10000, + "training_augmentation": "random_crop_32_padding_4_zero_then_horizontal_flip_p0.5", + "validation_class_counts": { + "0": 500, + "1": 500, + "2": 500, + "3": 500, + "4": 500, + "5": 500, + "6": 500, + "7": 500, + "8": 500, + "9": 500 + }, + "validation_examples": 5000, + "validation_index_sha256": "8328b206a97c420e49e54e3eca4abe3274c4756b084355784ea3fb8059e4515b" + }, + "timing": { + "apical_warmup_wall_s": 23.933976888656616, + "evaluation_wall_s": 0.36545491218566895, + "predictor_warmup_wall_s": 0.0, + "timing_excludes_data_loading_hashing_and_model_construction": true, + "total_timed_wall_s": 24.29965043067932, + "train_wall_s": 0.0 + }, + "work": { + "apical_macs_per_example": 202176, + "causal_scalar_observations": 800, + "components": { + "apical_projection_macs": 10238192640, + "apical_regression_macs": 10238192640, + "bp_reverse_macs_estimate": 0, + "local_weight_correlation_macs": 0, + "ordinary_forward_macs": 0, + "perturbation_forward_macs": 4107009331200, + "warmup_clean_forward_macs": 2053504665600 + }, + "definition": "multiply-accumulates in conv/linear maps; one local weight correlation equals one forward-weight MAC count; BP reverse is estimated as one weight-gradient plus one activation-gradient convolution per forward convolution; elementwise nonlinearities and optimizer arithmetic excluded", + "forward_macs_per_example": 40551040, + "logical_batch_loss_queries": 800, + "per_example_cross_entropy_terms": 101280, + "total_clean_forward_examples": 50640, + "total_forward_equivalent_examples": 151920, + "total_macs_estimate": 6180990382080 + } +} diff --git a/results/oral_a_v2_calibration/channel_subspace_etaA1.0.json b/results/oral_a_v2_calibration/channel_subspace_etaA1.0.json new file mode 100644 index 0000000..f000fa4 --- /dev/null +++ b/results/oral_a_v2_calibration/channel_subspace_etaA1.0.json @@ -0,0 +1,393 @@ +{ + "apical_warmup": { + "first": { + "calibration_mse": 0.667037273962917, + "parameter_update_rms": 0.023085309606734133, + "prediction_target_cosine": 0.0, + "target_power": 0.667037273962917 + }, + "last": { + "calibration_mse": 0.015573981316265417, + "parameter_update_rms": 0.005349574497052361, + "prediction_target_cosine": 0.004038878111100969, + "target_power": 0.014335632392748118 + }, + "mean": { + "calibration_mse": 0.2717085534018111, + "parameter_update_rms": 0.012732941480864037, + "prediction_target_cosine": -8.862524765940711e-05, + "target_power": 0.2708926473589623 + }, + "steps": 400 + }, + "architecture": { + "adaptive_apical_parameters": 390592, + "base_width": 16, + "blocks_per_stage": 3, + "bn_eps": 1e-05, + "bn_momentum": 0.1, + "depth": 20, + "family": "CIFAR 6n+2 ResNet, option-A shortcuts", + "fixed_traffic_coefficients": 188416, + "forward_parameters": 269722, + "hidden_shapes": [ + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ] + ], + "normalization": "batchnorm", + "predictor_parameters": 376832, + "residual_scale": 1.0, + "vectorizer_mode": "channel_gated", + "vectorizer_parameters": 13760 + }, + "args": { + "a_scale": 0.0, + "a_warmup_steps": 400, + "alignment_probe": 64, + "apical_calibration_mode": "channel_subspace", + "apical_seed": null, + "augment_train": 1, + "batch_size": 128, + "bn_eps": 1e-05, + "bn_momentum": 0.1, + "data_dir": "/home/yurenh2/sdrn/data", + "depth": 20, + "device": "cuda:0", + "epochs": 0, + "eta_A": 1.0, + "eta_P": 0.01, + "eval_every": 0, + "eval_split": "validation", + "learn_P": 0, + "loader_seed": 0, + "lr": 0.03, + "lr_gamma": 0.1, + "lr_milestones": "100,150", + "lr_schedule": "constant", + "max_steps": 0, + "mode": "sdil", + "momentum": 0.9, + "normalization": "batchnorm", + "nuisance_scale": 0.0, + "out": "results/oral_a_v2_calibration/channel_subspace_etaA1.0.json", + "output_lr": 0.1, + "pert_directions": 1, + "pert_every": 4, + "pert_sigma": 0.01, + "perturb_seed": 1000, + "predictor_warmup_steps": 0, + "residual_scale": null, + "seed": 0, + "split_seed": 2027, + "train_limit": 10000, + "use_residual": 1, + "val_examples": 5000, + "vectorizer_mode": "channel_gated", + "warmup_epochs": 0, + "weight_decay": 0.0001, + "weight_scale": 1.0, + "width": 16 + }, + "calibration_metric_space": "channel_basis_moments", + "counters": { + "apical_warmup_examples": 50640, + "calibration_event_examples": 50640, + "causal_scalar_observations": 800, + "logical_batch_loss_queries": 800, + "ordinary_examples": 0, + "per_example_loss_terms": 101280, + "perturbation_events": 400, + "perturbation_forward_examples": 101280, + "predictor_warmup_examples": 0 + }, + "diagnostics": { + "early_third_mean": 0.001714374249180158, + "innovation_negative_gradient_cosine": [ + -0.000828517833724618, + 0.0012920019216835499, + 0.005332308355718851, + 0.004775108303874731, + 0.0038991339970380068, + -0.004183789249509573, + 0.006285618990659714, + 0.003322963137179613, + 0.0036540841683745384, + 0.0062436810694634914, + -0.003957148641347885, + 0.006059933453798294, + 0.006437992211431265, + 0.021154729649424553, + 0.012715594843029976, + -0.02445901930332184, + -0.019053492695093155, + 0.021413326263427734, + 0.025747042149305344 + ], + "normalization_state": "training_batch_stats_without_running_update", + "raw_negative_gradient_cosine": [ + -0.000828517833724618, + 0.0012920019216835499, + 0.005332308355718851, + 0.004775108303874731, + 0.0038991339970380068, + -0.004183789249509573, + 0.006285618990659714, + 0.003322963137179613, + 0.0036540841683745384, + 0.0062436810694634914, + -0.003957148641347885, + 0.006059933453798294, + 0.006437992211431265, + 0.021154729649424553, + 0.012715594843029976, + -0.02445901930332184, + -0.019053492695093155, + 0.021413326263427734, + 0.025747042149305344 + ], + "teaching_negative_gradient_cosine": [ + -0.000828517833724618, + 0.0012920019216835499, + 0.005332308355718851, + 0.004775108303874731, + 0.0038991339970380068, + -0.004183789249509573, + 0.006285618990659714, + 0.003322963137179613, + 0.0036540841683745384, + 0.0062436810694634914, + -0.003957148641347885, + 0.006059933453798294, + 0.006437992211431265, + 0.021154729649424553, + 0.012715594843029976, + -0.02445901930332184, + -0.019053492695093155, + 0.021413326263427734, + 0.025747042149305344 + ], + "wall_s": 0.11605238914489746 + }, + "epochs": [], + "evaluation_protocol": { + "test_evaluations": 0, + "test_used_for_selection": false, + "validation_evaluations": 1 + }, + "final": { + "accuracy": 0.1004, + "epoch": 0, + "evaluation_split": "validation", + "finite": true, + "loss": 16.593759375, + "step": 0 + }, + "hardware": { + "cuda_device_name": "NVIDIA GeForce GTX 1080", + "cuda_visible_devices": "7", + "device": "cuda:0", + "device_total_memory_bytes": 8507949056, + "peak_memory_allocated_bytes": 1734868480, + "peak_memory_reserved_bytes": 2575302656, + "torch_version": "2.3.1+cu118" + }, + "protocol_family": "oral_a_cifar_local_resnet_development", + "provenance": { + "git_commit": "fc8fe99504fe86a3636721031a7dc3d41a7909a6", + "git_tracked_dirty": false + }, + "schema_version": 1, + "split": { + "cifar_source_files": [ + { + "bytes": 31035704, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_1", + "sha256": "54636561a3ce25bd3e19253c6b0d8538147b0ae398331ac4a2d86c6d987368cd" + }, + { + "bytes": 31035320, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_2", + "sha256": "766b2cef9fbc745cf056b3152224f7cf77163b330ea9a15f9392beb8b89bc5a8" + }, + { + "bytes": 31035999, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_3", + "sha256": "0f00d98ebfb30b3ec0ad19f9756dc2630b89003e10525f5e148445e82aa6a1f9" + }, + { + "bytes": 31035696, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_4", + "sha256": "3f7bb240661948b8f4d53e36ec720d8306f5668bd0071dcb4e6c947f78e9682b" + }, + { + "bytes": 31035623, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_5", + "sha256": "d91802434d8376bbaeeadf58a737e3a1b12ac839077e931237e0dcd43adcb154" + }, + { + "bytes": 31035526, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/test_batch", + "sha256": "f53d8d457504f7cff4ea9e021afcf0e0ad8e24a91f3fc42091b8adef61157831" + } + ], + "dataset": "cifar10", + "evaluation_split": "validation", + "input_layout": "NCHW", + "input_shape": [ + 3, + 32, + 32 + ], + "loader_seed": 0, + "normalization_mean": [ + 0.49140000343322754, + 0.4821999967098236, + 0.4465000033378601 + ], + "normalization_std": [ + 0.24699999392032623, + 0.2434999942779541, + 0.26159998774528503 + ], + "split_from_training_only": true, + "split_seed": 2027, + "test_examples": 10000, + "train_examples": 10000, + "training_augmentation": "random_crop_32_padding_4_zero_then_horizontal_flip_p0.5", + "validation_class_counts": { + "0": 500, + "1": 500, + "2": 500, + "3": 500, + "4": 500, + "5": 500, + "6": 500, + "7": 500, + "8": 500, + "9": 500 + }, + "validation_examples": 5000, + "validation_index_sha256": "8328b206a97c420e49e54e3eca4abe3274c4756b084355784ea3fb8059e4515b" + }, + "timing": { + "apical_warmup_wall_s": 23.875094175338745, + "evaluation_wall_s": 0.37113046646118164, + "predictor_warmup_wall_s": 0.0, + "timing_excludes_data_loading_hashing_and_model_construction": true, + "total_timed_wall_s": 24.24647092819214, + "train_wall_s": 0.0 + }, + "work": { + "apical_macs_per_example": 202176, + "causal_scalar_observations": 800, + "components": { + "apical_projection_macs": 10238192640, + "apical_regression_macs": 10238192640, + "bp_reverse_macs_estimate": 0, + "local_weight_correlation_macs": 0, + "ordinary_forward_macs": 0, + "perturbation_forward_macs": 4107009331200, + "warmup_clean_forward_macs": 2053504665600 + }, + "definition": "multiply-accumulates in conv/linear maps; one local weight correlation equals one forward-weight MAC count; BP reverse is estimated as one weight-gradient plus one activation-gradient convolution per forward convolution; elementwise nonlinearities and optimizer arithmetic excluded", + "forward_macs_per_example": 40551040, + "logical_batch_loss_queries": 800, + "per_example_cross_entropy_terms": 101280, + "total_clean_forward_examples": 50640, + "total_forward_equivalent_examples": 151920, + "total_macs_estimate": 6180990382080 + } +} diff --git a/results/oral_a_v2_calibration/unit_targets_etaA0.01.json b/results/oral_a_v2_calibration/unit_targets_etaA0.01.json new file mode 100644 index 0000000..b421acf --- /dev/null +++ b/results/oral_a_v2_calibration/unit_targets_etaA0.01.json @@ -0,0 +1,390 @@ +{ + "apical_warmup": { + "first": { + "calibration_mse": 4.1123504638671875, + "prediction_target_cosine": 0.0, + "target_power": 4.1123504638671875 + }, + "last": { + "calibration_mse": 60.77315496361774, + "prediction_target_cosine": 0.0006244406591329739, + "target_power": 60.77318373970363 + }, + "mean": { + "calibration_mse": 306.8068681861785, + "prediction_target_cosine": -1.0456776873674686e-06, + "target_power": 306.80684325786916 + }, + "steps": 400 + }, + "architecture": { + "adaptive_apical_parameters": 390592, + "base_width": 16, + "blocks_per_stage": 3, + "bn_eps": 1e-05, + "bn_momentum": 0.1, + "depth": 20, + "family": "CIFAR 6n+2 ResNet, option-A shortcuts", + "fixed_traffic_coefficients": 188416, + "forward_parameters": 269722, + "hidden_shapes": [ + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ] + ], + "normalization": "batchnorm", + "predictor_parameters": 376832, + "residual_scale": 1.0, + "vectorizer_mode": "channel_gated", + "vectorizer_parameters": 13760 + }, + "args": { + "a_scale": 0.0, + "a_warmup_steps": 400, + "alignment_probe": 64, + "apical_calibration_mode": "unit_targets", + "apical_seed": null, + "augment_train": 1, + "batch_size": 128, + "bn_eps": 1e-05, + "bn_momentum": 0.1, + "data_dir": "/home/yurenh2/sdrn/data", + "depth": 20, + "device": "cuda:0", + "epochs": 0, + "eta_A": 0.01, + "eta_P": 0.01, + "eval_every": 0, + "eval_split": "validation", + "learn_P": 0, + "loader_seed": 0, + "lr": 0.03, + "lr_gamma": 0.1, + "lr_milestones": "100,150", + "lr_schedule": "constant", + "max_steps": 0, + "mode": "sdil", + "momentum": 0.9, + "normalization": "batchnorm", + "nuisance_scale": 0.0, + "out": "results/oral_a_v2_calibration/unit_targets_etaA0.01.json", + "output_lr": 0.1, + "pert_directions": 1, + "pert_every": 4, + "pert_sigma": 0.01, + "perturb_seed": 1000, + "predictor_warmup_steps": 0, + "residual_scale": null, + "seed": 0, + "split_seed": 2027, + "train_limit": 10000, + "use_residual": 1, + "val_examples": 5000, + "vectorizer_mode": "channel_gated", + "warmup_epochs": 0, + "weight_decay": 0.0001, + "weight_scale": 1.0, + "width": 16 + }, + "calibration_metric_space": "full_hidden_field", + "counters": { + "apical_warmup_examples": 50640, + "calibration_event_examples": 50640, + "causal_scalar_observations": 800, + "logical_batch_loss_queries": 800, + "ordinary_examples": 0, + "per_example_loss_terms": 101280, + "perturbation_events": 400, + "perturbation_forward_examples": 101280, + "predictor_warmup_examples": 0 + }, + "diagnostics": { + "early_third_mean": 0.0011051012067279469, + "innovation_negative_gradient_cosine": [ + -0.00034784310264512897, + -0.0025916527956724167, + 2.2646039724349976e-05, + 0.007553194649517536, + 0.005484188441187143, + -0.003489925991743803, + -0.006299270782619715, + 0.006761397235095501, + -0.005489094648510218, + -0.012966052629053593, + 0.01738702319562435, + 0.004095735028386116, + 0.015238979831337929, + 0.026438727974891663, + -0.006385758984833956, + 0.02158680558204651, + 0.019882317632436752, + 0.046531397849321365, + 0.08820925652980804 + ], + "normalization_state": "training_batch_stats_without_running_update", + "raw_negative_gradient_cosine": [ + -0.00034784310264512897, + -0.0025916527956724167, + 2.2646039724349976e-05, + 0.007553194649517536, + 0.005484188441187143, + -0.003489925991743803, + -0.006299270782619715, + 0.006761397235095501, + -0.005489094648510218, + -0.012966052629053593, + 0.01738702319562435, + 0.004095735028386116, + 0.015238979831337929, + 0.026438727974891663, + -0.006385758984833956, + 0.02158680558204651, + 0.019882317632436752, + 0.046531397849321365, + 0.08820925652980804 + ], + "teaching_negative_gradient_cosine": [ + -0.00034784310264512897, + -0.0025916527956724167, + 2.2646039724349976e-05, + 0.007553194649517536, + 0.005484188441187143, + -0.003489925991743803, + -0.006299270782619715, + 0.006761397235095501, + -0.005489094648510218, + -0.012966052629053593, + 0.01738702319562435, + 0.004095735028386116, + 0.015238979831337929, + 0.026438727974891663, + -0.006385758984833956, + 0.02158680558204651, + 0.019882317632436752, + 0.046531397849321365, + 0.08820925652980804 + ], + "wall_s": 0.11040592193603516 + }, + "epochs": [], + "evaluation_protocol": { + "test_evaluations": 0, + "test_used_for_selection": false, + "validation_evaluations": 1 + }, + "final": { + "accuracy": 0.1004, + "epoch": 0, + "evaluation_split": "validation", + "finite": true, + "loss": 16.593759375, + "step": 0 + }, + "hardware": { + "cuda_device_name": "NVIDIA GeForce GTX 1080", + "cuda_visible_devices": "5", + "device": "cuda:0", + "device_total_memory_bytes": 8507949056, + "peak_memory_allocated_bytes": 1734868480, + "peak_memory_reserved_bytes": 2690646016, + "torch_version": "2.3.1+cu118" + }, + "protocol_family": "oral_a_cifar_local_resnet_development", + "provenance": { + "git_commit": "fc8fe99504fe86a3636721031a7dc3d41a7909a6", + "git_tracked_dirty": false + }, + "schema_version": 1, + "split": { + "cifar_source_files": [ + { + "bytes": 31035704, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_1", + "sha256": "54636561a3ce25bd3e19253c6b0d8538147b0ae398331ac4a2d86c6d987368cd" + }, + { + "bytes": 31035320, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_2", + "sha256": "766b2cef9fbc745cf056b3152224f7cf77163b330ea9a15f9392beb8b89bc5a8" + }, + { + "bytes": 31035999, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_3", + "sha256": "0f00d98ebfb30b3ec0ad19f9756dc2630b89003e10525f5e148445e82aa6a1f9" + }, + { + "bytes": 31035696, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_4", + "sha256": "3f7bb240661948b8f4d53e36ec720d8306f5668bd0071dcb4e6c947f78e9682b" + }, + { + "bytes": 31035623, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_5", + "sha256": "d91802434d8376bbaeeadf58a737e3a1b12ac839077e931237e0dcd43adcb154" + }, + { + "bytes": 31035526, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/test_batch", + "sha256": "f53d8d457504f7cff4ea9e021afcf0e0ad8e24a91f3fc42091b8adef61157831" + } + ], + "dataset": "cifar10", + "evaluation_split": "validation", + "input_layout": "NCHW", + "input_shape": [ + 3, + 32, + 32 + ], + "loader_seed": 0, + "normalization_mean": [ + 0.49140000343322754, + 0.4821999967098236, + 0.4465000033378601 + ], + "normalization_std": [ + 0.24699999392032623, + 0.2434999942779541, + 0.26159998774528503 + ], + "split_from_training_only": true, + "split_seed": 2027, + "test_examples": 10000, + "train_examples": 10000, + "training_augmentation": "random_crop_32_padding_4_zero_then_horizontal_flip_p0.5", + "validation_class_counts": { + "0": 500, + "1": 500, + "2": 500, + "3": 500, + "4": 500, + "5": 500, + "6": 500, + "7": 500, + "8": 500, + "9": 500 + }, + "validation_examples": 5000, + "validation_index_sha256": "8328b206a97c420e49e54e3eca4abe3274c4756b084355784ea3fb8059e4515b" + }, + "timing": { + "apical_warmup_wall_s": 24.429534912109375, + "evaluation_wall_s": 0.3710341453552246, + "predictor_warmup_wall_s": 0.0, + "timing_excludes_data_loading_hashing_and_model_construction": true, + "total_timed_wall_s": 24.80078649520874, + "train_wall_s": 0.0 + }, + "work": { + "apical_macs_per_example": 202176, + "causal_scalar_observations": 800, + "components": { + "apical_projection_macs": 10238192640, + "apical_regression_macs": 10238192640, + "bp_reverse_macs_estimate": 0, + "local_weight_correlation_macs": 0, + "ordinary_forward_macs": 0, + "perturbation_forward_macs": 4107009331200, + "warmup_clean_forward_macs": 2053504665600 + }, + "definition": "multiply-accumulates in conv/linear maps; one local weight correlation equals one forward-weight MAC count; BP reverse is estimated as one weight-gradient plus one activation-gradient convolution per forward convolution; elementwise nonlinearities and optimizer arithmetic excluded", + "forward_macs_per_example": 40551040, + "logical_batch_loss_queries": 800, + "per_example_cross_entropy_terms": 101280, + "total_clean_forward_examples": 50640, + "total_forward_equivalent_examples": 151920, + "total_macs_estimate": 6180990382080 + } +} diff --git a/results/oral_a_v2_calibration/unit_targets_etaA0.1.json b/results/oral_a_v2_calibration/unit_targets_etaA0.1.json new file mode 100644 index 0000000..7da80b1 --- /dev/null +++ b/results/oral_a_v2_calibration/unit_targets_etaA0.1.json @@ -0,0 +1,390 @@ +{ + "apical_warmup": { + "first": { + "calibration_mse": 4.1123504638671875, + "prediction_target_cosine": 0.0, + "target_power": 4.1123504638671875 + }, + "last": { + "calibration_mse": 60.77357632180919, + "prediction_target_cosine": 0.0003734676299838721, + "target_power": 60.77318373970363 + }, + "mean": { + "calibration_mse": 306.80736262691147, + "prediction_target_cosine": -8.709214493754583e-06, + "target_power": 306.80684325786916 + }, + "steps": 400 + }, + "architecture": { + "adaptive_apical_parameters": 390592, + "base_width": 16, + "blocks_per_stage": 3, + "bn_eps": 1e-05, + "bn_momentum": 0.1, + "depth": 20, + "family": "CIFAR 6n+2 ResNet, option-A shortcuts", + "fixed_traffic_coefficients": 188416, + "forward_parameters": 269722, + "hidden_shapes": [ + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ] + ], + "normalization": "batchnorm", + "predictor_parameters": 376832, + "residual_scale": 1.0, + "vectorizer_mode": "channel_gated", + "vectorizer_parameters": 13760 + }, + "args": { + "a_scale": 0.0, + "a_warmup_steps": 400, + "alignment_probe": 64, + "apical_calibration_mode": "unit_targets", + "apical_seed": null, + "augment_train": 1, + "batch_size": 128, + "bn_eps": 1e-05, + "bn_momentum": 0.1, + "data_dir": "/home/yurenh2/sdrn/data", + "depth": 20, + "device": "cuda:0", + "epochs": 0, + "eta_A": 0.1, + "eta_P": 0.01, + "eval_every": 0, + "eval_split": "validation", + "learn_P": 0, + "loader_seed": 0, + "lr": 0.03, + "lr_gamma": 0.1, + "lr_milestones": "100,150", + "lr_schedule": "constant", + "max_steps": 0, + "mode": "sdil", + "momentum": 0.9, + "normalization": "batchnorm", + "nuisance_scale": 0.0, + "out": "results/oral_a_v2_calibration/unit_targets_etaA0.1.json", + "output_lr": 0.1, + "pert_directions": 1, + "pert_every": 4, + "pert_sigma": 0.01, + "perturb_seed": 1000, + "predictor_warmup_steps": 0, + "residual_scale": null, + "seed": 0, + "split_seed": 2027, + "train_limit": 10000, + "use_residual": 1, + "val_examples": 5000, + "vectorizer_mode": "channel_gated", + "warmup_epochs": 0, + "weight_decay": 0.0001, + "weight_scale": 1.0, + "width": 16 + }, + "calibration_metric_space": "full_hidden_field", + "counters": { + "apical_warmup_examples": 50640, + "calibration_event_examples": 50640, + "causal_scalar_observations": 800, + "logical_batch_loss_queries": 800, + "ordinary_examples": 0, + "per_example_loss_terms": 101280, + "perturbation_events": 400, + "perturbation_forward_examples": 101280, + "predictor_warmup_examples": 0 + }, + "diagnostics": { + "early_third_mean": -0.0007603219904315969, + "innovation_negative_gradient_cosine": [ + -0.0026482653338462114, + -0.0014026857679709792, + -0.0032926220446825027, + 0.003918738104403019, + 0.006381125655025244, + -0.00751822255551815, + -3.789784386754036e-06, + 0.007395345717668533, + 0.0014983054716140032, + -0.00718093104660511, + 0.018063008785247803, + -0.006030107382684946, + 0.01317618228495121, + 0.04094434902071953, + 0.011970864608883858, + 0.0008559874258935452, + 0.03555452078580856, + 0.03732839599251747, + 0.038500867784023285 + ], + "normalization_state": "training_batch_stats_without_running_update", + "raw_negative_gradient_cosine": [ + -0.0026482653338462114, + -0.0014026857679709792, + -0.0032926220446825027, + 0.003918738104403019, + 0.006381125655025244, + -0.00751822255551815, + -3.789784386754036e-06, + 0.007395345717668533, + 0.0014983054716140032, + -0.00718093104660511, + 0.018063008785247803, + -0.006030107382684946, + 0.01317618228495121, + 0.04094434902071953, + 0.011970864608883858, + 0.0008559874258935452, + 0.03555452078580856, + 0.03732839599251747, + 0.038500867784023285 + ], + "teaching_negative_gradient_cosine": [ + -0.0026482653338462114, + -0.0014026857679709792, + -0.0032926220446825027, + 0.003918738104403019, + 0.006381125655025244, + -0.00751822255551815, + -3.789784386754036e-06, + 0.007395345717668533, + 0.0014983054716140032, + -0.00718093104660511, + 0.018063008785247803, + -0.006030107382684946, + 0.01317618228495121, + 0.04094434902071953, + 0.011970864608883858, + 0.0008559874258935452, + 0.03555452078580856, + 0.03732839599251747, + 0.038500867784023285 + ], + "wall_s": 0.11375069618225098 + }, + "epochs": [], + "evaluation_protocol": { + "test_evaluations": 0, + "test_used_for_selection": false, + "validation_evaluations": 1 + }, + "final": { + "accuracy": 0.1004, + "epoch": 0, + "evaluation_split": "validation", + "finite": true, + "loss": 16.593759375, + "step": 0 + }, + "hardware": { + "cuda_device_name": "NVIDIA GeForce GTX 1080", + "cuda_visible_devices": "7", + "device": "cuda:0", + "device_total_memory_bytes": 8507949056, + "peak_memory_allocated_bytes": 1734868480, + "peak_memory_reserved_bytes": 2690646016, + "torch_version": "2.3.1+cu118" + }, + "protocol_family": "oral_a_cifar_local_resnet_development", + "provenance": { + "git_commit": "fc8fe99504fe86a3636721031a7dc3d41a7909a6", + "git_tracked_dirty": false + }, + "schema_version": 1, + "split": { + "cifar_source_files": [ + { + "bytes": 31035704, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_1", + "sha256": "54636561a3ce25bd3e19253c6b0d8538147b0ae398331ac4a2d86c6d987368cd" + }, + { + "bytes": 31035320, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_2", + "sha256": "766b2cef9fbc745cf056b3152224f7cf77163b330ea9a15f9392beb8b89bc5a8" + }, + { + "bytes": 31035999, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_3", + "sha256": "0f00d98ebfb30b3ec0ad19f9756dc2630b89003e10525f5e148445e82aa6a1f9" + }, + { + "bytes": 31035696, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_4", + "sha256": "3f7bb240661948b8f4d53e36ec720d8306f5668bd0071dcb4e6c947f78e9682b" + }, + { + "bytes": 31035623, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_5", + "sha256": "d91802434d8376bbaeeadf58a737e3a1b12ac839077e931237e0dcd43adcb154" + }, + { + "bytes": 31035526, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/test_batch", + "sha256": "f53d8d457504f7cff4ea9e021afcf0e0ad8e24a91f3fc42091b8adef61157831" + } + ], + "dataset": "cifar10", + "evaluation_split": "validation", + "input_layout": "NCHW", + "input_shape": [ + 3, + 32, + 32 + ], + "loader_seed": 0, + "normalization_mean": [ + 0.49140000343322754, + 0.4821999967098236, + 0.4465000033378601 + ], + "normalization_std": [ + 0.24699999392032623, + 0.2434999942779541, + 0.26159998774528503 + ], + "split_from_training_only": true, + "split_seed": 2027, + "test_examples": 10000, + "train_examples": 10000, + "training_augmentation": "random_crop_32_padding_4_zero_then_horizontal_flip_p0.5", + "validation_class_counts": { + "0": 500, + "1": 500, + "2": 500, + "3": 500, + "4": 500, + "5": 500, + "6": 500, + "7": 500, + "8": 500, + "9": 500 + }, + "validation_examples": 5000, + "validation_index_sha256": "8328b206a97c420e49e54e3eca4abe3274c4756b084355784ea3fb8059e4515b" + }, + "timing": { + "apical_warmup_wall_s": 24.465684175491333, + "evaluation_wall_s": 0.37288689613342285, + "predictor_warmup_wall_s": 0.0, + "timing_excludes_data_loading_hashing_and_model_construction": true, + "total_timed_wall_s": 24.83878993988037, + "train_wall_s": 0.0 + }, + "work": { + "apical_macs_per_example": 202176, + "causal_scalar_observations": 800, + "components": { + "apical_projection_macs": 10238192640, + "apical_regression_macs": 10238192640, + "bp_reverse_macs_estimate": 0, + "local_weight_correlation_macs": 0, + "ordinary_forward_macs": 0, + "perturbation_forward_macs": 4107009331200, + "warmup_clean_forward_macs": 2053504665600 + }, + "definition": "multiply-accumulates in conv/linear maps; one local weight correlation equals one forward-weight MAC count; BP reverse is estimated as one weight-gradient plus one activation-gradient convolution per forward convolution; elementwise nonlinearities and optimizer arithmetic excluded", + "forward_macs_per_example": 40551040, + "logical_batch_loss_queries": 800, + "per_example_cross_entropy_terms": 101280, + "total_clean_forward_examples": 50640, + "total_forward_equivalent_examples": 151920, + "total_macs_estimate": 6180990382080 + } +} diff --git a/results/oral_a_v2_calibration/unit_targets_etaA1.0.json b/results/oral_a_v2_calibration/unit_targets_etaA1.0.json new file mode 100644 index 0000000..a648c1b --- /dev/null +++ b/results/oral_a_v2_calibration/unit_targets_etaA1.0.json @@ -0,0 +1,390 @@ +{ + "apical_warmup": { + "first": { + "calibration_mse": 4.1123504638671875, + "prediction_target_cosine": 0.0, + "target_power": 4.1123504638671875 + }, + "last": { + "calibration_mse": 60.785771079685375, + "prediction_target_cosine": 0.00018066191971056846, + "target_power": 60.77318373970363 + }, + "mean": { + "calibration_mse": 306.8130476901478, + "prediction_target_cosine": 1.1424882244632998e-06, + "target_power": 306.80684325786916 + }, + "steps": 400 + }, + "architecture": { + "adaptive_apical_parameters": 390592, + "base_width": 16, + "blocks_per_stage": 3, + "bn_eps": 1e-05, + "bn_momentum": 0.1, + "depth": 20, + "family": "CIFAR 6n+2 ResNet, option-A shortcuts", + "fixed_traffic_coefficients": 188416, + "forward_parameters": 269722, + "hidden_shapes": [ + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ] + ], + "normalization": "batchnorm", + "predictor_parameters": 376832, + "residual_scale": 1.0, + "vectorizer_mode": "channel_gated", + "vectorizer_parameters": 13760 + }, + "args": { + "a_scale": 0.0, + "a_warmup_steps": 400, + "alignment_probe": 64, + "apical_calibration_mode": "unit_targets", + "apical_seed": null, + "augment_train": 1, + "batch_size": 128, + "bn_eps": 1e-05, + "bn_momentum": 0.1, + "data_dir": "/home/yurenh2/sdrn/data", + "depth": 20, + "device": "cuda:0", + "epochs": 0, + "eta_A": 1.0, + "eta_P": 0.01, + "eval_every": 0, + "eval_split": "validation", + "learn_P": 0, + "loader_seed": 0, + "lr": 0.03, + "lr_gamma": 0.1, + "lr_milestones": "100,150", + "lr_schedule": "constant", + "max_steps": 0, + "mode": "sdil", + "momentum": 0.9, + "normalization": "batchnorm", + "nuisance_scale": 0.0, + "out": "results/oral_a_v2_calibration/unit_targets_etaA1.0.json", + "output_lr": 0.1, + "pert_directions": 1, + "pert_every": 4, + "pert_sigma": 0.01, + "perturb_seed": 1000, + "predictor_warmup_steps": 0, + "residual_scale": null, + "seed": 0, + "split_seed": 2027, + "train_limit": 10000, + "use_residual": 1, + "val_examples": 5000, + "vectorizer_mode": "channel_gated", + "warmup_epochs": 0, + "weight_decay": 0.0001, + "weight_scale": 1.0, + "width": 16 + }, + "calibration_metric_space": "full_hidden_field", + "counters": { + "apical_warmup_examples": 50640, + "calibration_event_examples": 50640, + "causal_scalar_observations": 800, + "logical_batch_loss_queries": 800, + "ordinary_examples": 0, + "per_example_loss_terms": 101280, + "perturbation_events": 400, + "perturbation_forward_examples": 101280, + "predictor_warmup_examples": 0 + }, + "diagnostics": { + "early_third_mean": -0.0031450521055376157, + "innovation_negative_gradient_cosine": [ + -0.0038595087826251984, + -0.0002608323993626982, + -0.0040445514023303986, + -0.001344867399893701, + -0.001432035001926124, + -0.007928517647087574, + -0.004752017557621002, + 0.009463246911764145, + -0.0019811633974313736, + -0.005859529599547386, + 0.01658852957189083, + -0.009742104448378086, + 0.005521662067621946, + 0.04838317632675171, + 0.024612773209810257, + -0.0016448963433504105, + -0.001705970149487257, + -0.004393921699374914, + 0.0048246243968605995 + ], + "normalization_state": "training_batch_stats_without_running_update", + "raw_negative_gradient_cosine": [ + -0.0038595087826251984, + -0.0002608323993626982, + -0.0040445514023303986, + -0.001344867399893701, + -0.001432035001926124, + -0.007928517647087574, + -0.004752017557621002, + 0.009463246911764145, + -0.0019811633974313736, + -0.005859529599547386, + 0.01658852957189083, + -0.009742104448378086, + 0.005521662067621946, + 0.04838317632675171, + 0.024612773209810257, + -0.0016448963433504105, + -0.001705970149487257, + -0.004393921699374914, + 0.0048246243968605995 + ], + "teaching_negative_gradient_cosine": [ + -0.0038595087826251984, + -0.0002608323993626982, + -0.0040445514023303986, + -0.001344867399893701, + -0.001432035001926124, + -0.007928517647087574, + -0.004752017557621002, + 0.009463246911764145, + -0.0019811633974313736, + -0.005859529599547386, + 0.01658852957189083, + -0.009742104448378086, + 0.005521662067621946, + 0.04838317632675171, + 0.024612773209810257, + -0.0016448963433504105, + -0.001705970149487257, + -0.004393921699374914, + 0.0048246243968605995 + ], + "wall_s": 0.11835074424743652 + }, + "epochs": [], + "evaluation_protocol": { + "test_evaluations": 0, + "test_used_for_selection": false, + "validation_evaluations": 1 + }, + "final": { + "accuracy": 0.1004, + "epoch": 0, + "evaluation_split": "validation", + "finite": true, + "loss": 16.593759375, + "step": 0 + }, + "hardware": { + "cuda_device_name": "NVIDIA GeForce GTX 1080", + "cuda_visible_devices": "5", + "device": "cuda:0", + "device_total_memory_bytes": 8507949056, + "peak_memory_allocated_bytes": 1734868480, + "peak_memory_reserved_bytes": 2690646016, + "torch_version": "2.3.1+cu118" + }, + "protocol_family": "oral_a_cifar_local_resnet_development", + "provenance": { + "git_commit": "fc8fe99504fe86a3636721031a7dc3d41a7909a6", + "git_tracked_dirty": false + }, + "schema_version": 1, + "split": { + "cifar_source_files": [ + { + "bytes": 31035704, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_1", + "sha256": "54636561a3ce25bd3e19253c6b0d8538147b0ae398331ac4a2d86c6d987368cd" + }, + { + "bytes": 31035320, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_2", + "sha256": "766b2cef9fbc745cf056b3152224f7cf77163b330ea9a15f9392beb8b89bc5a8" + }, + { + "bytes": 31035999, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_3", + "sha256": "0f00d98ebfb30b3ec0ad19f9756dc2630b89003e10525f5e148445e82aa6a1f9" + }, + { + "bytes": 31035696, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_4", + "sha256": "3f7bb240661948b8f4d53e36ec720d8306f5668bd0071dcb4e6c947f78e9682b" + }, + { + "bytes": 31035623, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_5", + "sha256": "d91802434d8376bbaeeadf58a737e3a1b12ac839077e931237e0dcd43adcb154" + }, + { + "bytes": 31035526, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/test_batch", + "sha256": "f53d8d457504f7cff4ea9e021afcf0e0ad8e24a91f3fc42091b8adef61157831" + } + ], + "dataset": "cifar10", + "evaluation_split": "validation", + "input_layout": "NCHW", + "input_shape": [ + 3, + 32, + 32 + ], + "loader_seed": 0, + "normalization_mean": [ + 0.49140000343322754, + 0.4821999967098236, + 0.4465000033378601 + ], + "normalization_std": [ + 0.24699999392032623, + 0.2434999942779541, + 0.26159998774528503 + ], + "split_from_training_only": true, + "split_seed": 2027, + "test_examples": 10000, + "train_examples": 10000, + "training_augmentation": "random_crop_32_padding_4_zero_then_horizontal_flip_p0.5", + "validation_class_counts": { + "0": 500, + "1": 500, + "2": 500, + "3": 500, + "4": 500, + "5": 500, + "6": 500, + "7": 500, + "8": 500, + "9": 500 + }, + "validation_examples": 5000, + "validation_index_sha256": "8328b206a97c420e49e54e3eca4abe3274c4756b084355784ea3fb8059e4515b" + }, + "timing": { + "apical_warmup_wall_s": 24.51742196083069, + "evaluation_wall_s": 0.3782050609588623, + "predictor_warmup_wall_s": 0.0, + "timing_excludes_data_loading_hashing_and_model_construction": true, + "total_timed_wall_s": 24.89581036567688, + "train_wall_s": 0.0 + }, + "work": { + "apical_macs_per_example": 202176, + "causal_scalar_observations": 800, + "components": { + "apical_projection_macs": 10238192640, + "apical_regression_macs": 10238192640, + "bp_reverse_macs_estimate": 0, + "local_weight_correlation_macs": 0, + "ordinary_forward_macs": 0, + "perturbation_forward_macs": 4107009331200, + "warmup_clean_forward_macs": 2053504665600 + }, + "definition": "multiply-accumulates in conv/linear maps; one local weight correlation equals one forward-weight MAC count; BP reverse is estimated as one weight-gradient plus one activation-gradient convolution per forward convolution; elementwise nonlinearities and optimizer arithmetic excluded", + "forward_macs_per_example": 40551040, + "logical_batch_loss_queries": 800, + "per_example_cross_entropy_terms": 101280, + "total_clean_forward_examples": 50640, + "total_forward_equivalent_examples": 151920, + "total_macs_estimate": 6180990382080 + } +} diff --git a/results/oral_a_v2_calibration_gate.json b/results/oral_a_v2_calibration_gate.json new file mode 100644 index 0000000..1ffc605 --- /dev/null +++ b/results/oral_a_v2_calibration_gate.json @@ -0,0 +1,147 @@ +{ + "checks": { + "all_six_records_finite": true, + "both_modes_selected": true, + "structured_all_layer_at_least_0.01": true, + "structured_early_gain_over_unit_at_least_0.01": false, + "structured_early_third_at_least_0.01": false + }, + "confirmation_test_seeds_touched": false, + "protocol": "oral_a_v2_causal_capture_v1", + "review_score_after": 5, + "review_score_before": 5, + "rows": [ + { + "calibration_mode": "channel_subspace", + "eta_A": 0.01, + "finite": true, + "metric_space": "channel_basis_moments", + "metrics": { + "all_layer_alignment": 0.05273995646520665, + "early_third_alignment": 0.007209055746595065, + "mean_calibration_mse": 0.2708950854477958, + "mean_parameter_update_rms": 0.011981565658686305, + "mean_prediction_target_cosine": 0.00018159585958168355, + "mean_target_power": 0.2708926473589623 + }, + "path": "results/oral_a_v2_calibration/channel_subspace_etaA0.01.json", + "source_commit": "fc8fe99504fe86a3636721031a7dc3d41a7909a6" + }, + { + "calibration_mode": "channel_subspace", + "eta_A": 0.1, + "finite": true, + "metric_space": "channel_basis_moments", + "metrics": { + "all_layer_alignment": 0.026189661456124957, + "early_third_alignment": 0.0038381816702894866, + "mean_calibration_mse": 0.27095736647643465, + "mean_parameter_update_rms": 0.012069313096218597, + "mean_prediction_target_cosine": 0.00010544442092045836, + "mean_target_power": 0.2708926473589623 + }, + "path": "results/oral_a_v2_calibration/channel_subspace_etaA0.1.json", + "source_commit": "fc8fe99504fe86a3636721031a7dc3d41a7909a6" + }, + { + "calibration_mode": "channel_subspace", + "eta_A": 1.0, + "finite": true, + "metric_space": "channel_basis_moments", + "metrics": { + "all_layer_alignment": 0.003992186883758558, + "early_third_alignment": 0.001714374249180158, + "mean_calibration_mse": 0.2717085534018111, + "mean_parameter_update_rms": 0.012732941480864037, + "mean_prediction_target_cosine": -8.862524765940711e-05, + "mean_target_power": 0.2708926473589623 + }, + "path": "results/oral_a_v2_calibration/channel_subspace_etaA1.0.json", + "source_commit": "fc8fe99504fe86a3636721031a7dc3d41a7909a6" + }, + { + "calibration_mode": "unit_targets", + "eta_A": 0.01, + "finite": true, + "metric_space": "full_hidden_field", + "metrics": { + "all_layer_alignment": 0.011664319529173602, + "early_third_alignment": 0.0011051012067279469, + "mean_calibration_mse": 306.8068681861785, + "mean_parameter_update_rms": 0.0, + "mean_prediction_target_cosine": -1.0456776873674686e-06, + "mean_target_power": 306.80684325786916 + }, + "path": "results/oral_a_v2_calibration/unit_targets_etaA0.01.json", + "source_commit": "fc8fe99504fe86a3636721031a7dc3d41a7909a6" + }, + { + "calibration_mode": "unit_targets", + "eta_A": 0.1, + "finite": true, + "metric_space": "full_hidden_field", + "metrics": { + "all_layer_alignment": 0.00986900356426639, + "early_third_alignment": -0.0007603219904315969, + "mean_calibration_mse": 306.80736262691147, + "mean_parameter_update_rms": 0.0, + "mean_prediction_target_cosine": -8.709214493754583e-06, + "mean_target_power": 306.80684325786916 + }, + "path": "results/oral_a_v2_calibration/unit_targets_etaA0.1.json", + "source_commit": "fc8fe99504fe86a3636721031a7dc3d41a7909a6" + }, + { + "calibration_mode": "unit_targets", + "eta_A": 1.0, + "finite": true, + "metric_space": "full_hidden_field", + "metrics": { + "all_layer_alignment": 0.0031812682450675454, + "early_third_alignment": -0.0031450521055376157, + "mean_calibration_mse": 306.8130476901478, + "mean_parameter_update_rms": 0.0, + "mean_prediction_target_cosine": 1.1424882244632998e-06, + "mean_target_power": 306.80684325786916 + }, + "path": "results/oral_a_v2_calibration/unit_targets_etaA1.0.json", + "source_commit": "fc8fe99504fe86a3636721031a7dc3d41a7909a6" + } + ], + "score_change_rule": "mechanics/calibration alone cannot raise score", + "selected": { + "channel_subspace": { + "calibration_mode": "channel_subspace", + "eta_A": 0.01, + "finite": true, + "metric_space": "channel_basis_moments", + "metrics": { + "all_layer_alignment": 0.05273995646520665, + "early_third_alignment": 0.007209055746595065, + "mean_calibration_mse": 0.2708950854477958, + "mean_parameter_update_rms": 0.011981565658686305, + "mean_prediction_target_cosine": 0.00018159585958168355, + "mean_target_power": 0.2708926473589623 + }, + "path": "results/oral_a_v2_calibration/channel_subspace_etaA0.01.json", + "source_commit": "fc8fe99504fe86a3636721031a7dc3d41a7909a6" + }, + "unit_targets": { + "calibration_mode": "unit_targets", + "eta_A": 0.01, + "finite": true, + "metric_space": "full_hidden_field", + "metrics": { + "all_layer_alignment": 0.011664319529173602, + "early_third_alignment": 0.0011051012067279469, + "mean_calibration_mse": 306.8068681861785, + "mean_parameter_update_rms": 0.0, + "mean_prediction_target_cosine": -1.0456776873674686e-06, + "mean_target_power": 306.80684325786916 + }, + "path": "results/oral_a_v2_calibration/unit_targets_etaA0.01.json", + "source_commit": "fc8fe99504fe86a3636721031a7dc3d41a7909a6" + } + }, + "status": "failed" +} |