diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 14:30:04 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 14:33:51 -0500 |
| commit | 39d1839ead0709798c2eb07104c3764ccf1ed3ae (patch) | |
| tree | d8813f14d590d3b9ec4368bcac4ee78584957a9d | |
| parent | 41048d9d9c399ded6f5c473b578f59e6ac375ae9 (diff) | |
results: pass reciprocal KP short gate
| -rw-r--r-- | BASELINES.md | 4 | ||||
| -rw-r--r-- | KP_BASELINE.md | 21 | ||||
| -rw-r--r-- | RESULTS.md | 22 | ||||
| -rw-r--r-- | REVIEW_SCORECARD.md | 2 | ||||
| -rw-r--r-- | ROADMAP.md | 9 | ||||
| -rw-r--r-- | results/kp_short/kp.json | 736 | ||||
| -rw-r--r-- | results/kp_short_gate.json | 35 |
7 files changed, 828 insertions, 1 deletions
diff --git a/BASELINES.md b/BASELINES.md index 14c969c..767ca93 100644 --- a/BASELINES.md +++ b/BASELINES.md @@ -37,7 +37,9 @@ The audited implementations and completed results are in `RESULTS.md`: correlation from their own local activity pair and use matched optimizer and decay dynamics. The feedback update never reads W or its update. A separate reciprocal correlation is charged in the MAC audit; `KP_BASELINE.md` freezes - its staged ResNet gate before any KP task endpoint; + its staged ResNet gate before any KP task endpoint. The frozen KP-1 record + passes at 82.66% validation accuracy versus matched BP's epoch-20 81.02%, + with 0.886 early alignment and 1.326x BP MACs; - direct node perturbation uses a causal target on every hidden update and no learned vectorizer; - the no-traffic, `P=0` SDIL backbone is learned direct feedback by node diff --git a/KP_BASELINE.md b/KP_BASELINE.md index 25696f7..78bb756 100644 --- a/KP_BASELINE.md +++ b/KP_BASELINE.md @@ -76,3 +76,24 @@ the reviewer score. A KP-2 pass only opens a separately frozen raw-versus-norm-matched-raw-versus-innovation mixed-traffic experiment. The somato-dendritic innovation must be load-bearing there to affect the SDIL paper assessment. + +## Audited KP-1 outcome + +The sole frozen record from source revision `8f420f2` passes every gate: + +| metric | KP-1 | frozen requirement/reference | +|:--|--:|--:| +| epoch-20 validation accuracy | **82.66%** | at least 70%; BP epoch 20 is 81.02% | +| early-third teaching alignment | 0.885583 | at least 0.50 | +| final feedback/forward cosine | 0.901034 | at least 0.80 | +| epoch-11--20 mean feedback cosine | 0.854594 | at least 0.70 | +| MAC ratio to matched BP | 1.3261x | at most 1.40x | +| task-loss queries for feedback | 0 | exactly 0 | + +All epoch losses and tracking diagnostics are finite. The run uses +`1.451874816e14` MACs, 2.163 GB peak allocated memory, and 558.1 s on a GTX +1080. Its final mean parameter relative error is still 0.445, so KP does not +require exact weight equality to obtain a strong teaching direction. Unlike +the failed intermittent-mirror trajectory, these alignment gates cover the +constant-LR training interval rather than only a decayed-LR endpoint. KP-2 is +opened without changing any optimizer or feedback setting. @@ -868,6 +868,28 @@ so the single RRM-3 full validation run opens. The 9.86-point gap despite nearly perfect instantaneous alignment is retained as a negative diagnostic rather than treating cosine as a proxy for optimization equivalence. +### Modified Kolen--Pollack reciprocal baseline + +The independently implemented Akrout et al. reciprocal-plasticity baseline +uses the same local parent activity and child teaching field at corresponding +forward and feedback synapses, but each path recomputes its own correlation. +It never reads or copies a weight or weight-update tensor. Mechanics checks +give exactly zero direction, post-observation independence, and symmetric- +limit update errors. + +Its sole frozen 20-epoch full-development record passes every KP-1 gate: + +| method | validation accuracy | early alignment | train-period feedback cosine | estimated MACs | +|:--|--:|--:|--:|--:| +| KP | **82.66%** | 0.885583 | 0.854594 (epochs 11--20) | 1.4519e14 | +| BP at epoch 20 | 81.02% | n/a | exact by construction | 1.0949e14 | + +KP's final feedback/forward cosine is 0.901034. It uses zero task-loss queries, +costs `1.3261x` matched BP after explicitly charging the second reciprocal +correlation, and remains finite throughout the constant-LR screen. The result +opens the frozen KP-2 full baseline. It is entirely prior-art baseline evidence +and does not change the SDIL reviewer score. + ## How to run `experiments/run.py --mode {bp,fa,dfa,sdil} --dataset {mnist,fmnist,cifar10} --depth D --residual {0,1} --act {tanh,gelu,silu,relu}` Batteries: `experiments/run_v2.sh <ds> "<depths>" <res> <act> "<seeds>" <ep> <pfx>`. diff --git a/REVIEW_SCORECARD.md b/REVIEW_SCORECARD.md index e9eae6a..e6e57d7 100644 --- a/REVIEW_SCORECARD.md +++ b/REVIEW_SCORECARD.md @@ -66,6 +66,7 @@ Every formal result report records: | Normalized response mirror short gate fails | 5 | WM reaches 64.04% and 0.939 early alignment at 0.9968x BP MACs, but misses its two accuracy gates by under one point | Strong inherited baseline and useful warning that alignment is insufficient; full run and any SDIL scale claim remain closed | | Residual response mirror capture passes | 5 | RRM reaches 0.665 early and 0.730 all-layer alignment with 0.954 feedback/forward cosine after 20 zero-query local observations | Opens the frozen short accuracy gate, but remains inherited predictive weight estimation rather than SDIL evidence | | Residual response mirror short gate passes | 5 | RRM reaches 65.08%, within 9.86 points of matched BP, at 0.9969x BP MACs and nearly exact alignment | Opens one full validation run; the inherited baseline still supplies no Harnett-specific evidence | +| Modified KP short gate passes | 5 | KP reaches 82.66% versus BP's epoch-20 81.02%, with 0.886 early alignment and 1.326x BP MACs | Strongly solves the substrate at useful scale, but all credit belongs to inherited reciprocal plasticity until innovation is load-bearing | | Oral-A A4 | not opened | The prerequisite A3 gate failed | No oral-A confirmation claim is available | These are conditional reviewer forecasts, not promised scores. A failed stage leaves its negative @@ -90,6 +91,7 @@ retroactively reopened by success on standard vision benchmarks. | 2026-07-22 / response-mirror WM-2 | Two clean short ResNet records; selected WM reaches 64.04% with 0.9393 early alignment and 0.9968x BP MACs, missing both accuracy gates narrowly | 5 → 5 | Substantially strengthens the comparator and cost story, but closes its full run and demonstrates that high credit cosine alone is not a scale result | | 2026-07-22 / residual response-mirror RRM-1 | Four frozen-forward records; selected etaM 0.1 reaches 0.6651 early and 0.7303 all-layer alignment with controlled norms, zero task-loss queries, and 2.4458e9 MACs | 5 → 5 | Residual prediction removes much of the fixed-point estimator noise and opens the short task gate, but the gain belongs to an inherited baseline and does not establish somato-dendritic innovation | | 2026-07-22 / residual response-mirror RRM-2 | Two clean short ResNet records; selected RRM reaches 65.08%, 9.86 points below BP, with 0.9991 early alignment and 0.9969x BP MACs | 5 → 5 | Narrowly opens the full baseline run and strengthens the efficient comparator; the remaining accuracy gap despite near-exact direction warns that alignment is not trajectory equivalence | +| 2026-07-22 / modified KP-1 | One clean constant-LR ResNet record reaches 82.66%, above BP's epoch-20 81.02%, with 0.8856 early alignment, 0.8546 train-period feedback cosine, zero queries, and 1.326x BP MACs | 5 → 5 | Establishes a stable strong substrate under a gate that observes the training trajectory, but reciprocal Kolen--Pollack plasticity is prior art and adds no Harnett-specific evidence | Future rows are appended only after an audited frozen stage. A score staying flat is informative: engineering, theory exposition, or visualization may make the paper more defensible without @@ -456,6 +456,15 @@ feedback tracking. No KP task endpoint had been generated when this protocol was committed. KP is inherited Akrout et al. machinery and cannot raise the paper score; only a later load-bearing innovation ablation can do that. +**Modified Kolen--Pollack KP-1 status: passed.** The sole frozen record reaches +`82.66%` validation accuracy versus matched BP's epoch-20 `81.02%`, with early +alignment `0.885583`, final feedback/forward cosine `0.901034`, and epoch-11--20 +mean cosine `0.854594`. It is finite, uses zero task-loss queries, and costs +`1.3261x` BP after charging reciprocal correlations. KP-2 is opened at exactly +the frozen settings. This strongly repairs the feedback-substrate engineering +problem but remains inherited Akrout et al. evidence, so the paper score stays +5/10. + Prepare convolutional local-update primitives and ResNet-20/32/56 protocols early. Queue frozen runs opportunistically on authorized idle GPUs. Because BurstCCN already reports CIFAR-10 and ImageNet scaling, dataset scale alone is not novel. The oral-level target is a memorable joint diff --git a/results/kp_short/kp.json b/results/kp_short/kp.json new file mode 100644 index 0000000..79215fb --- /dev/null +++ b/results/kp_short/kp.json @@ -0,0 +1,736 @@ +{ + "architecture": { + "adaptive_apical_parameters": 0, + "adaptive_feedback_parameters": 267904, + "base_width": 16, + "blocks_per_stage": 3, + "bn_eps": 1e-05, + "bn_momentum": 0.1, + "depth": 20, + "family": "CIFAR 6n+2 ResNet, option-A shortcuts", + "fixed_feedback_parameters": 0, + "fixed_traffic_coefficients": 0, + "forward_parameters": 269722, + "hidden_shapes": [ + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 16, + 32, + 32 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 32, + 16, + 16 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ], + [ + 64, + 8, + 8 + ] + ], + "normalization": "batchnorm", + "predictor_parameters": 0, + "residual_scale": 1.0, + "vectorizer_mode": null, + "vectorizer_parameters": 0 + }, + "args": { + "a_scale": 1.0, + "a_warmup_steps": 0, + "alignment_probe": 32, + "apical_calibration_mode": "unit_targets", + "apical_seed": null, + "augment_train": 1, + "batch_size": 128, + "bn_eps": 1e-05, + "bn_momentum": 0.1, + "data_dir": "/home/yurenh2/sdrn/data", + "depth": 20, + "device": "cuda", + "epochs": 20, + "eta_A": 0.01, + "eta_P": 0.01, + "eval_every": 0, + "eval_split": "validation", + "learn_P": 0, + "loader_seed": 0, + "lr": 0.1, + "lr_gamma": 0.1, + "lr_milestones": "100,150", + "lr_schedule": "step", + "max_steps": 0, + "mirror_batch_size": 1, + "mirror_eta": 0.1, + "mirror_every": 16, + "mirror_noise_std": 1.0, + "mirror_seed": 3000, + "mirror_warmup_steps": 0, + "mode": "kp", + "momentum": 0.9, + "normalization": "batchnorm", + "nuisance_scale": 0.0, + "out": "results/kp_short/kp.json", + "output_lr": 0.1, + "pert_directions": 1, + "pert_every": 4, + "pert_sigma": 0.01, + "perturb_seed": 1000, + "predictor_warmup_steps": 0, + "residual_scale": null, + "seed": 0, + "split_seed": 2027, + "train_limit": 0, + "use_residual": 1, + "val_examples": 5000, + "vectorizer_mode": "spatial_template", + "warmup_epochs": 0, + "weight_decay": 0.0001, + "weight_scale": 1.0, + "width": 16 + }, + "calibration_metric_space": "reciprocal_local_activity_products", + "counters": { + "apical_warmup_examples": 0, + "calibration_event_examples": 0, + "causal_scalar_observations": 0, + "logical_batch_loss_queries": 0, + "mirror_conv_examples": 0, + "mirror_events": 0, + "mirror_readout_examples": 0, + "ordinary_examples": 900000, + "per_example_loss_terms": 0, + "perturbation_events": 0, + "perturbation_forward_examples": 0, + "predictor_warmup_examples": 0 + }, + "diagnostics": { + "early_third_mean": 0.8855825364589691, + "feedback_forward_cosine": [ + 0.9264333248138428, + 0.9220218658447266, + 0.9038588404655457, + 0.8975453972816467, + 0.9213621616363525, + 0.8933338522911072, + 0.8915912508964539, + 0.9265133142471313, + 0.9061511754989624, + 0.8942975997924805, + 0.9047679901123047, + 0.8851061463356018, + 0.8851138353347778, + 0.9171422719955444, + 0.9053213000297546, + 0.8831733465194702, + 0.8767958879470825, + 0.7925058007240295, + 0.9866083860397339 + ], + "feedback_forward_norm_ratio": [ + 0.9885096549987793, + 1.0436758995056152, + 0.9917821884155273, + 1.058516263961792, + 1.0159763097763062, + 1.0456026792526245, + 1.008077621459961, + 1.0261797904968262, + 1.0106136798858643, + 1.0411553382873535, + 1.0132758617401123, + 1.0498096942901611, + 1.0225006341934204, + 1.0299837589263916, + 1.011872410774231, + 1.0414197444915771, + 1.0175153017044067, + 1.0444023609161377, + 1.090652585029602 + ], + "feedback_forward_relative_error": [ + 0.3815428614616394, + 0.4058021008968353, + 0.43677207827568054, + 0.46938687562942505, + 0.4000545144081116, + 0.47448983788490295, + 0.46758344769477844, + 0.3892383575439453, + 0.43566325306892395, + 0.4709555506706238, + 0.4395098090171814, + 0.49367427825927734, + 0.48523032665252686, + 0.4142261743545532, + 0.4378886818885803, + 0.49502187967300415, + 0.5010300874710083, + 0.6598381400108337, + 0.1934662163257599 + ], + "innovation_negative_gradient_cosine": [ + 0.8906962275505066, + 0.8764338493347168, + 0.8894688487052917, + 0.8657850027084351, + 0.8935482501983643, + 0.8975630402565002, + 0.9018489122390747, + 0.9003725051879883, + 0.9135095477104187, + 0.8963901996612549, + 0.9183286428451538, + 0.9015354514122009, + 0.9177113771438599, + 0.9407275915145874, + 0.9454699754714966, + 0.9456664323806763, + 0.9667372107505798, + 0.9770752787590027, + 0.9845573902130127 + ], + "mean_feedback_forward_cosine": 0.9010338814635026, + "mean_feedback_forward_relative_error": 0.44480918269408376, + "normalization_state": "training_batch_stats_without_running_update", + "raw_negative_gradient_cosine": [ + 0.8906962275505066, + 0.8764338493347168, + 0.8894688487052917, + 0.8657850027084351, + 0.8935482501983643, + 0.8975630402565002, + 0.9018489122390747, + 0.9003725051879883, + 0.9135095477104187, + 0.8963901996612549, + 0.9183286428451538, + 0.9015354514122009, + 0.9177113771438599, + 0.9407275915145874, + 0.9454699754714966, + 0.9456664323806763, + 0.9667372107505798, + 0.9770752787590027, + 0.9845573902130127 + ], + "teaching_negative_gradient_cosine": [ + 0.8906962275505066, + 0.8764338493347168, + 0.8894688487052917, + 0.8657850027084351, + 0.8935482501983643, + 0.8975630402565002, + 0.9018489122390747, + 0.9003725051879883, + 0.9135095477104187, + 0.8963901996612549, + 0.9183286428451538, + 0.9015354514122009, + 0.9177113771438599, + 0.9407275915145874, + 0.9454699754714966, + 0.9456664323806763, + 0.9667372107505798, + 0.9770752787590027, + 0.9845573902130127 + ], + "wall_s": 0.1368110179901123 + }, + "epochs": [ + { + "epoch": 1, + "feedback_tracking": { + "max_feedback_forward_relative_error": 1.319318175315857, + "mean_feedback_forward_cosine": 0.26153333798835154, + "mean_feedback_forward_relative_error": 1.2144870224751925, + "min_feedback_forward_cosine": 0.1322619915008545 + }, + "lr": 0.1, + "output_lr": 0.1, + "step": 352, + "train_examples": 45000, + "train_loss": 1.831467347378201 + }, + { + "epoch": 2, + "feedback_tracking": { + "max_feedback_forward_relative_error": 1.2504557371139526, + "mean_feedback_forward_cosine": 0.40932882773248774, + "mean_feedback_forward_relative_error": 1.0874902041334855, + "min_feedback_forward_cosine": 0.22793743014335632 + }, + "lr": 0.1, + "output_lr": 0.1, + "step": 704, + "train_examples": 45000, + "train_loss": 1.366863529311286 + }, + { + "epoch": 3, + "feedback_tracking": { + "max_feedback_forward_relative_error": 1.19105064868927, + "mean_feedback_forward_cosine": 0.5041392759272927, + "mean_feedback_forward_relative_error": 0.9971762236795927, + "min_feedback_forward_cosine": 0.30597037076950073 + }, + "lr": 0.1, + "output_lr": 0.1, + "step": 1056, + "train_examples": 45000, + "train_loss": 1.066310328886244 + }, + { + "epoch": 4, + "feedback_tracking": { + "max_feedback_forward_relative_error": 1.1481163501739502, + "mean_feedback_forward_cosine": 0.5715355308432328, + "mean_feedback_forward_relative_error": 0.9270231566931072, + "min_feedback_forward_cosine": 0.3590141832828522 + }, + "lr": 0.1, + "output_lr": 0.1, + "step": 1408, + "train_examples": 45000, + "train_loss": 0.8910715007040235 + }, + { + "epoch": 5, + "feedback_tracking": { + "max_feedback_forward_relative_error": 1.1075339317321777, + "mean_feedback_forward_cosine": 0.62295485170264, + "mean_feedback_forward_relative_error": 0.8701311349868774, + "min_feedback_forward_cosine": 0.40619510412216187 + }, + "lr": 0.1, + "output_lr": 0.1, + "step": 1760, + "train_examples": 45000, + "train_loss": 0.7831426847881741 + }, + { + "epoch": 6, + "feedback_tracking": { + "max_feedback_forward_relative_error": 1.0683236122131348, + "mean_feedback_forward_cosine": 0.6647130376414249, + "mean_feedback_forward_relative_error": 0.8211206464391005, + "min_feedback_forward_cosine": 0.44973599910736084 + }, + "lr": 0.1, + "output_lr": 0.1, + "step": 2112, + "train_examples": 45000, + "train_loss": 0.7067856804741753 + }, + { + "epoch": 7, + "feedback_tracking": { + "max_feedback_forward_relative_error": 1.0321979522705078, + "mean_feedback_forward_cosine": 0.6994027605182246, + "mean_feedback_forward_relative_error": 0.7776816581424913, + "min_feedback_forward_cosine": 0.487300843000412 + }, + "lr": 0.1, + "output_lr": 0.1, + "step": 2464, + "train_examples": 45000, + "train_loss": 0.6526998540454441 + }, + { + "epoch": 8, + "feedback_tracking": { + "max_feedback_forward_relative_error": 0.9973180890083313, + "mean_feedback_forward_cosine": 0.7283705755283958, + "mean_feedback_forward_relative_error": 0.7394031474464818, + "min_feedback_forward_cosine": 0.5232464075088501 + }, + "lr": 0.1, + "output_lr": 0.1, + "step": 2816, + "train_examples": 45000, + "train_loss": 0.6056941405296326 + }, + { + "epoch": 9, + "feedback_tracking": { + "max_feedback_forward_relative_error": 0.9644169211387634, + "mean_feedback_forward_cosine": 0.7529234384235582, + "mean_feedback_forward_relative_error": 0.705177787103151, + "min_feedback_forward_cosine": 0.5548381805419922 + }, + "lr": 0.1, + "output_lr": 0.1, + "step": 3168, + "train_examples": 45000, + "train_loss": 0.5672190690252517 + }, + { + "epoch": 10, + "feedback_tracking": { + "max_feedback_forward_relative_error": 0.9325018525123596, + "mean_feedback_forward_cosine": 0.7752530323831659, + "mean_feedback_forward_relative_error": 0.6724935864147387, + "min_feedback_forward_cosine": 0.5845517516136169 + }, + "lr": 0.1, + "output_lr": 0.1, + "step": 3520, + "train_examples": 45000, + "train_loss": 0.5412863729900784 + }, + { + "epoch": 11, + "feedback_tracking": { + "max_feedback_forward_relative_error": 0.9008499383926392, + "mean_feedback_forward_cosine": 0.7944865289487337, + "mean_feedback_forward_relative_error": 0.6431144601420352, + "min_feedback_forward_cosine": 0.6125654578208923 + }, + "lr": 0.1, + "output_lr": 0.1, + "step": 3872, + "train_examples": 45000, + "train_loss": 0.5144231326527066 + }, + { + "epoch": 12, + "feedback_tracking": { + "max_feedback_forward_relative_error": 0.870076596736908, + "mean_feedback_forward_cosine": 0.8117237467514841, + "mean_feedback_forward_relative_error": 0.6155103241142473, + "min_feedback_forward_cosine": 0.6389982104301453 + }, + "lr": 0.1, + "output_lr": 0.1, + "step": 4224, + "train_examples": 45000, + "train_loss": 0.4893017247200012 + }, + { + "epoch": 13, + "feedback_tracking": { + "max_feedback_forward_relative_error": 0.8421436548233032, + "mean_feedback_forward_cosine": 0.8270191926705209, + "mean_feedback_forward_relative_error": 0.5898554764295879, + "min_feedback_forward_cosine": 0.662228524684906 + }, + "lr": 0.1, + "output_lr": 0.1, + "step": 4576, + "train_examples": 45000, + "train_loss": 0.4666704408009847 + }, + { + "epoch": 14, + "feedback_tracking": { + "max_feedback_forward_relative_error": 0.8131178021430969, + "mean_feedback_forward_cosine": 0.8408146061395344, + "mean_feedback_forward_relative_error": 0.5656935533410624, + "min_feedback_forward_cosine": 0.6854491233825684 + }, + "lr": 0.1, + "output_lr": 0.1, + "step": 4928, + "train_examples": 45000, + "train_loss": 0.4554117670377096 + }, + { + "epoch": 15, + "feedback_tracking": { + "max_feedback_forward_relative_error": 0.7860043048858643, + "mean_feedback_forward_cosine": 0.8532820532196447, + "mean_feedback_forward_relative_error": 0.5429034452689322, + "min_feedback_forward_cosine": 0.7059769034385681 + }, + "lr": 0.1, + "output_lr": 0.1, + "step": 5280, + "train_examples": 45000, + "train_loss": 0.437559742252032 + }, + { + "epoch": 16, + "feedback_tracking": { + "max_feedback_forward_relative_error": 0.7597430944442749, + "mean_feedback_forward_cosine": 0.8647166176846153, + "mean_feedback_forward_relative_error": 0.5210003123471612, + "min_feedback_forward_cosine": 0.7249040603637695 + }, + "lr": 0.1, + "output_lr": 0.1, + "step": 5632, + "train_examples": 45000, + "train_loss": 0.4244806206067403 + }, + { + "epoch": 17, + "feedback_tracking": { + "max_feedback_forward_relative_error": 0.7334027886390686, + "mean_feedback_forward_cosine": 0.8751521863435444, + "mean_feedback_forward_relative_error": 0.5003075152635574, + "min_feedback_forward_cosine": 0.7437110543251038 + }, + "lr": 0.1, + "output_lr": 0.1, + "step": 5984, + "train_examples": 45000, + "train_loss": 0.4090740834448073 + }, + { + "epoch": 18, + "feedback_tracking": { + "max_feedback_forward_relative_error": 0.7081427574157715, + "mean_feedback_forward_cosine": 0.8844657948142604, + "mean_feedback_forward_relative_error": 0.4810000712934293, + "min_feedback_forward_cosine": 0.7609646320343018 + }, + "lr": 0.1, + "output_lr": 0.1, + "step": 6336, + "train_examples": 45000, + "train_loss": 0.39670369214481777 + }, + { + "epoch": 19, + "feedback_tracking": { + "max_feedback_forward_relative_error": 0.6829538345336914, + "mean_feedback_forward_cosine": 0.8932415654784754, + "mean_feedback_forward_relative_error": 0.4622263304497066, + "min_feedback_forward_cosine": 0.7776626348495483 + }, + "lr": 0.1, + "output_lr": 0.1, + "step": 6688, + "train_examples": 45000, + "train_loss": 0.3911194394800398 + }, + { + "epoch": 20, + "feedback_tracking": { + "max_feedback_forward_relative_error": 0.6598381400108337, + "mean_feedback_forward_cosine": 0.9010338814635026, + "mean_feedback_forward_relative_error": 0.44480918269408376, + "min_feedback_forward_cosine": 0.7925058007240295 + }, + "lr": 0.1, + "output_lr": 0.1, + "step": 7040, + "train_examples": 45000, + "train_loss": 0.3764604419231415 + } + ], + "evaluation_protocol": { + "test_evaluations": 0, + "test_used_for_selection": false, + "validation_evaluations": 1 + }, + "final": { + "accuracy": 0.8266, + "epoch": 20, + "evaluation_split": "validation", + "finite": true, + "loss": 0.5224007934570313, + "step": 7040 + }, + "hardware": { + "cuda_device_name": "NVIDIA GeForce GTX 1080", + "cuda_visible_devices": "7", + "device": "cuda", + "device_total_memory_bytes": 8507949056, + "peak_memory_allocated_bytes": 2162790400, + "peak_memory_reserved_bytes": 2866806784, + "torch_version": "2.3.1+cu118" + }, + "protocol_family": "oral_a_cifar_local_resnet_development", + "provenance": { + "git_commit": "8f420f256aef160d79e3a9620a92ae156f06ef74", + "git_tracked_dirty": false + }, + "schema_version": 1, + "split": { + "cifar_source_files": [ + { + "bytes": 31035704, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_1", + "sha256": "54636561a3ce25bd3e19253c6b0d8538147b0ae398331ac4a2d86c6d987368cd" + }, + { + "bytes": 31035320, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_2", + "sha256": "766b2cef9fbc745cf056b3152224f7cf77163b330ea9a15f9392beb8b89bc5a8" + }, + { + "bytes": 31035999, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_3", + "sha256": "0f00d98ebfb30b3ec0ad19f9756dc2630b89003e10525f5e148445e82aa6a1f9" + }, + { + "bytes": 31035696, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_4", + "sha256": "3f7bb240661948b8f4d53e36ec720d8306f5668bd0071dcb4e6c947f78e9682b" + }, + { + "bytes": 31035623, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_5", + "sha256": "d91802434d8376bbaeeadf58a737e3a1b12ac839077e931237e0dcd43adcb154" + }, + { + "bytes": 31035526, + "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/test_batch", + "sha256": "f53d8d457504f7cff4ea9e021afcf0e0ad8e24a91f3fc42091b8adef61157831" + } + ], + "dataset": "cifar10", + "evaluation_split": "validation", + "input_layout": "NCHW", + "input_shape": [ + 3, + 32, + 32 + ], + "loader_seed": 0, + "normalization_mean": [ + 0.49140000343322754, + 0.4821999967098236, + 0.4465000033378601 + ], + "normalization_std": [ + 0.24699999392032623, + 0.2434999942779541, + 0.26159998774528503 + ], + "split_from_training_only": true, + "split_seed": 2027, + "test_examples": 10000, + "train_examples": 45000, + "training_augmentation": "random_crop_32_padding_4_zero_then_horizontal_flip_p0.5", + "validation_class_counts": { + "0": 500, + "1": 500, + "2": 500, + "3": 500, + "4": 500, + "5": 500, + "6": 500, + "7": 500, + "8": 500, + "9": 500 + }, + "validation_examples": 5000, + "validation_index_sha256": "8328b206a97c420e49e54e3eca4abe3274c4756b084355784ea3fb8059e4515b" + }, + "timing": { + "apical_warmup_wall_s": 0.0, + "evaluation_wall_s": 0.36439943313598633, + "mirror_warmup_wall_s": 0.0, + "predictor_warmup_wall_s": 0.0, + "timing_excludes_data_loading_hashing_and_model_construction": true, + "total_timed_wall_s": 558.1345100402832, + "train_wall_s": 557.5914545059204 + }, + "work": { + "apical_macs_per_example": 40108672, + "causal_scalar_observations": 0, + "components": { + "apical_projection_macs": 36097804800000, + "apical_regression_macs": 0, + "bp_reverse_macs_estimate": 0, + "kp_reciprocal_correlation_macs": 36097804800000, + "local_weight_correlation_macs": 36495936000000, + "mirror_feedback_prediction_macs": 0, + "mirror_local_correlation_macs": 0, + "mirror_response_macs": 0, + "ordinary_forward_macs": 36495936000000, + "perturbation_forward_macs": 0, + "warmup_clean_forward_macs": 0 + }, + "definition": "multiply-accumulates in conv/linear maps; one local weight correlation equals one forward-weight MAC count; BP reverse is estimated as one weight-gradient plus one activation-gradient convolution per forward convolution; elementwise nonlinearities and optimizer arithmetic excluded", + "forward_macs_per_example": 40551040, + "logical_batch_loss_queries": 0, + "mirror_probe_examples": 0, + "mirror_readout_probe_examples": 0, + "per_example_cross_entropy_terms": 0, + "total_clean_forward_examples": 900000, + "total_forward_equivalent_examples": 900000, + "total_macs_estimate": 145187481600000 + } +} diff --git a/results/kp_short_gate.json b/results/kp_short_gate.json new file mode 100644 index 0000000..22e5b27 --- /dev/null +++ b/results/kp_short_gate.json @@ -0,0 +1,35 @@ +{ + "checks": { + "accuracy_at_least_0.70": true, + "early_alignment_at_least_0.50": true, + "epoch11_to20_feedback_cosine_at_least_0.70": true, + "final_feedback_cosine_at_least_0.80": true, + "macs_at_most_1.40x_bp": true, + "record_and_trajectory_finite": true, + "within_15_points_of_bp_epoch20": true, + "zero_task_loss_queries": true + }, + "confirmation_test_seeds_touched": false, + "full_validation_opened": true, + "metrics": { + "accuracy": 0.8266, + "bp_epoch20_accuracy": 0.8102, + "bp_epoch20_total_macs": 109487808000000, + "early_third_alignment": 0.8855825364589691, + "epoch11_to20_mean_feedback_forward_cosine": 0.8545936173514315, + "final_mean_feedback_forward_cosine": 0.9010338814635026, + "final_mean_feedback_forward_relative_error": 0.44480918269408376, + "logical_batch_loss_queries": 0, + "loss": 0.5224007934570313, + "mac_ratio_to_bp": 1.3260607208430002, + "peak_memory_allocated_bytes": 2162790400, + "source_commit": "8f420f256aef160d79e3a9620a92ae156f06ef74", + "total_macs": 145187481600000, + "wall_s": 558.1345100402832 + }, + "protocol": "kolen_pollack_short_v1", + "review_score_after": 5, + "review_score_before": 5, + "score_change_rule": "inherited KP baseline cannot raise score", + "status": "passed" +} |
