summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorYurenHao0426 <Blackhao0426@gmail.com>2026-07-22 14:30:04 -0500
committerYurenHao0426 <Blackhao0426@gmail.com>2026-07-22 14:33:51 -0500
commit39d1839ead0709798c2eb07104c3764ccf1ed3ae (patch)
treed8813f14d590d3b9ec4368bcac4ee78584957a9d
parent41048d9d9c399ded6f5c473b578f59e6ac375ae9 (diff)
results: pass reciprocal KP short gate
-rw-r--r--BASELINES.md4
-rw-r--r--KP_BASELINE.md21
-rw-r--r--RESULTS.md22
-rw-r--r--REVIEW_SCORECARD.md2
-rw-r--r--ROADMAP.md9
-rw-r--r--results/kp_short/kp.json736
-rw-r--r--results/kp_short_gate.json35
7 files changed, 828 insertions, 1 deletions
diff --git a/BASELINES.md b/BASELINES.md
index 14c969c..767ca93 100644
--- a/BASELINES.md
+++ b/BASELINES.md
@@ -37,7 +37,9 @@ The audited implementations and completed results are in `RESULTS.md`:
correlation from their own local activity pair and use matched optimizer and
decay dynamics. The feedback update never reads W or its update. A separate
reciprocal correlation is charged in the MAC audit; `KP_BASELINE.md` freezes
- its staged ResNet gate before any KP task endpoint;
+ its staged ResNet gate before any KP task endpoint. The frozen KP-1 record
+ passes at 82.66% validation accuracy versus matched BP's epoch-20 81.02%,
+ with 0.886 early alignment and 1.326x BP MACs;
- direct node perturbation uses a causal target on every hidden update and no
learned vectorizer;
- the no-traffic, `P=0` SDIL backbone is learned direct feedback by node
diff --git a/KP_BASELINE.md b/KP_BASELINE.md
index 25696f7..78bb756 100644
--- a/KP_BASELINE.md
+++ b/KP_BASELINE.md
@@ -76,3 +76,24 @@ the reviewer score. A KP-2 pass only opens a separately frozen
raw-versus-norm-matched-raw-versus-innovation mixed-traffic experiment. The
somato-dendritic innovation must be load-bearing there to affect the SDIL
paper assessment.
+
+## Audited KP-1 outcome
+
+The sole frozen record from source revision `8f420f2` passes every gate:
+
+| metric | KP-1 | frozen requirement/reference |
+|:--|--:|--:|
+| epoch-20 validation accuracy | **82.66%** | at least 70%; BP epoch 20 is 81.02% |
+| early-third teaching alignment | 0.885583 | at least 0.50 |
+| final feedback/forward cosine | 0.901034 | at least 0.80 |
+| epoch-11--20 mean feedback cosine | 0.854594 | at least 0.70 |
+| MAC ratio to matched BP | 1.3261x | at most 1.40x |
+| task-loss queries for feedback | 0 | exactly 0 |
+
+All epoch losses and tracking diagnostics are finite. The run uses
+`1.451874816e14` MACs, 2.163 GB peak allocated memory, and 558.1 s on a GTX
+1080. Its final mean parameter relative error is still 0.445, so KP does not
+require exact weight equality to obtain a strong teaching direction. Unlike
+the failed intermittent-mirror trajectory, these alignment gates cover the
+constant-LR training interval rather than only a decayed-LR endpoint. KP-2 is
+opened without changing any optimizer or feedback setting.
diff --git a/RESULTS.md b/RESULTS.md
index a4c63c9..76ffc29 100644
--- a/RESULTS.md
+++ b/RESULTS.md
@@ -868,6 +868,28 @@ so the single RRM-3 full validation run opens. The 9.86-point gap despite nearly
perfect instantaneous alignment is retained as a negative diagnostic rather
than treating cosine as a proxy for optimization equivalence.
+### Modified Kolen--Pollack reciprocal baseline
+
+The independently implemented Akrout et al. reciprocal-plasticity baseline
+uses the same local parent activity and child teaching field at corresponding
+forward and feedback synapses, but each path recomputes its own correlation.
+It never reads or copies a weight or weight-update tensor. Mechanics checks
+give exactly zero direction, post-observation independence, and symmetric-
+limit update errors.
+
+Its sole frozen 20-epoch full-development record passes every KP-1 gate:
+
+| method | validation accuracy | early alignment | train-period feedback cosine | estimated MACs |
+|:--|--:|--:|--:|--:|
+| KP | **82.66%** | 0.885583 | 0.854594 (epochs 11--20) | 1.4519e14 |
+| BP at epoch 20 | 81.02% | n/a | exact by construction | 1.0949e14 |
+
+KP's final feedback/forward cosine is 0.901034. It uses zero task-loss queries,
+costs `1.3261x` matched BP after explicitly charging the second reciprocal
+correlation, and remains finite throughout the constant-LR screen. The result
+opens the frozen KP-2 full baseline. It is entirely prior-art baseline evidence
+and does not change the SDIL reviewer score.
+
## How to run
`experiments/run.py --mode {bp,fa,dfa,sdil} --dataset {mnist,fmnist,cifar10} --depth D --residual {0,1} --act {tanh,gelu,silu,relu}`
Batteries: `experiments/run_v2.sh <ds> "<depths>" <res> <act> "<seeds>" <ep> <pfx>`.
diff --git a/REVIEW_SCORECARD.md b/REVIEW_SCORECARD.md
index e9eae6a..e6e57d7 100644
--- a/REVIEW_SCORECARD.md
+++ b/REVIEW_SCORECARD.md
@@ -66,6 +66,7 @@ Every formal result report records:
| Normalized response mirror short gate fails | 5 | WM reaches 64.04% and 0.939 early alignment at 0.9968x BP MACs, but misses its two accuracy gates by under one point | Strong inherited baseline and useful warning that alignment is insufficient; full run and any SDIL scale claim remain closed |
| Residual response mirror capture passes | 5 | RRM reaches 0.665 early and 0.730 all-layer alignment with 0.954 feedback/forward cosine after 20 zero-query local observations | Opens the frozen short accuracy gate, but remains inherited predictive weight estimation rather than SDIL evidence |
| Residual response mirror short gate passes | 5 | RRM reaches 65.08%, within 9.86 points of matched BP, at 0.9969x BP MACs and nearly exact alignment | Opens one full validation run; the inherited baseline still supplies no Harnett-specific evidence |
+| Modified KP short gate passes | 5 | KP reaches 82.66% versus BP's epoch-20 81.02%, with 0.886 early alignment and 1.326x BP MACs | Strongly solves the substrate at useful scale, but all credit belongs to inherited reciprocal plasticity until innovation is load-bearing |
| Oral-A A4 | not opened | The prerequisite A3 gate failed | No oral-A confirmation claim is available |
These are conditional reviewer forecasts, not promised scores. A failed stage leaves its negative
@@ -90,6 +91,7 @@ retroactively reopened by success on standard vision benchmarks.
| 2026-07-22 / response-mirror WM-2 | Two clean short ResNet records; selected WM reaches 64.04% with 0.9393 early alignment and 0.9968x BP MACs, missing both accuracy gates narrowly | 5 → 5 | Substantially strengthens the comparator and cost story, but closes its full run and demonstrates that high credit cosine alone is not a scale result |
| 2026-07-22 / residual response-mirror RRM-1 | Four frozen-forward records; selected etaM 0.1 reaches 0.6651 early and 0.7303 all-layer alignment with controlled norms, zero task-loss queries, and 2.4458e9 MACs | 5 → 5 | Residual prediction removes much of the fixed-point estimator noise and opens the short task gate, but the gain belongs to an inherited baseline and does not establish somato-dendritic innovation |
| 2026-07-22 / residual response-mirror RRM-2 | Two clean short ResNet records; selected RRM reaches 65.08%, 9.86 points below BP, with 0.9991 early alignment and 0.9969x BP MACs | 5 → 5 | Narrowly opens the full baseline run and strengthens the efficient comparator; the remaining accuracy gap despite near-exact direction warns that alignment is not trajectory equivalence |
+| 2026-07-22 / modified KP-1 | One clean constant-LR ResNet record reaches 82.66%, above BP's epoch-20 81.02%, with 0.8856 early alignment, 0.8546 train-period feedback cosine, zero queries, and 1.326x BP MACs | 5 → 5 | Establishes a stable strong substrate under a gate that observes the training trajectory, but reciprocal Kolen--Pollack plasticity is prior art and adds no Harnett-specific evidence |
Future rows are appended only after an audited frozen stage. A score staying flat is informative:
engineering, theory exposition, or visualization may make the paper more defensible without
diff --git a/ROADMAP.md b/ROADMAP.md
index ddb674d..b2fed59 100644
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@ -456,6 +456,15 @@ feedback tracking. No KP task endpoint had been generated when this protocol
was committed. KP is inherited Akrout et al. machinery and cannot raise the
paper score; only a later load-bearing innovation ablation can do that.
+**Modified Kolen--Pollack KP-1 status: passed.** The sole frozen record reaches
+`82.66%` validation accuracy versus matched BP's epoch-20 `81.02%`, with early
+alignment `0.885583`, final feedback/forward cosine `0.901034`, and epoch-11--20
+mean cosine `0.854594`. It is finite, uses zero task-loss queries, and costs
+`1.3261x` BP after charging reciprocal correlations. KP-2 is opened at exactly
+the frozen settings. This strongly repairs the feedback-substrate engineering
+problem but remains inherited Akrout et al. evidence, so the paper score stays
+5/10.
+
Prepare convolutional local-update primitives and ResNet-20/32/56 protocols early. Queue frozen
runs opportunistically on authorized idle GPUs. Because BurstCCN already reports CIFAR-10 and
ImageNet scaling, dataset scale alone is not novel. The oral-level target is a memorable joint
diff --git a/results/kp_short/kp.json b/results/kp_short/kp.json
new file mode 100644
index 0000000..79215fb
--- /dev/null
+++ b/results/kp_short/kp.json
@@ -0,0 +1,736 @@
+{
+ "architecture": {
+ "adaptive_apical_parameters": 0,
+ "adaptive_feedback_parameters": 267904,
+ "base_width": 16,
+ "blocks_per_stage": 3,
+ "bn_eps": 1e-05,
+ "bn_momentum": 0.1,
+ "depth": 20,
+ "family": "CIFAR 6n+2 ResNet, option-A shortcuts",
+ "fixed_feedback_parameters": 0,
+ "fixed_traffic_coefficients": 0,
+ "forward_parameters": 269722,
+ "hidden_shapes": [
+ [
+ 16,
+ 32,
+ 32
+ ],
+ [
+ 16,
+ 32,
+ 32
+ ],
+ [
+ 16,
+ 32,
+ 32
+ ],
+ [
+ 16,
+ 32,
+ 32
+ ],
+ [
+ 16,
+ 32,
+ 32
+ ],
+ [
+ 16,
+ 32,
+ 32
+ ],
+ [
+ 16,
+ 32,
+ 32
+ ],
+ [
+ 32,
+ 16,
+ 16
+ ],
+ [
+ 32,
+ 16,
+ 16
+ ],
+ [
+ 32,
+ 16,
+ 16
+ ],
+ [
+ 32,
+ 16,
+ 16
+ ],
+ [
+ 32,
+ 16,
+ 16
+ ],
+ [
+ 32,
+ 16,
+ 16
+ ],
+ [
+ 64,
+ 8,
+ 8
+ ],
+ [
+ 64,
+ 8,
+ 8
+ ],
+ [
+ 64,
+ 8,
+ 8
+ ],
+ [
+ 64,
+ 8,
+ 8
+ ],
+ [
+ 64,
+ 8,
+ 8
+ ],
+ [
+ 64,
+ 8,
+ 8
+ ]
+ ],
+ "normalization": "batchnorm",
+ "predictor_parameters": 0,
+ "residual_scale": 1.0,
+ "vectorizer_mode": null,
+ "vectorizer_parameters": 0
+ },
+ "args": {
+ "a_scale": 1.0,
+ "a_warmup_steps": 0,
+ "alignment_probe": 32,
+ "apical_calibration_mode": "unit_targets",
+ "apical_seed": null,
+ "augment_train": 1,
+ "batch_size": 128,
+ "bn_eps": 1e-05,
+ "bn_momentum": 0.1,
+ "data_dir": "/home/yurenh2/sdrn/data",
+ "depth": 20,
+ "device": "cuda",
+ "epochs": 20,
+ "eta_A": 0.01,
+ "eta_P": 0.01,
+ "eval_every": 0,
+ "eval_split": "validation",
+ "learn_P": 0,
+ "loader_seed": 0,
+ "lr": 0.1,
+ "lr_gamma": 0.1,
+ "lr_milestones": "100,150",
+ "lr_schedule": "step",
+ "max_steps": 0,
+ "mirror_batch_size": 1,
+ "mirror_eta": 0.1,
+ "mirror_every": 16,
+ "mirror_noise_std": 1.0,
+ "mirror_seed": 3000,
+ "mirror_warmup_steps": 0,
+ "mode": "kp",
+ "momentum": 0.9,
+ "normalization": "batchnorm",
+ "nuisance_scale": 0.0,
+ "out": "results/kp_short/kp.json",
+ "output_lr": 0.1,
+ "pert_directions": 1,
+ "pert_every": 4,
+ "pert_sigma": 0.01,
+ "perturb_seed": 1000,
+ "predictor_warmup_steps": 0,
+ "residual_scale": null,
+ "seed": 0,
+ "split_seed": 2027,
+ "train_limit": 0,
+ "use_residual": 1,
+ "val_examples": 5000,
+ "vectorizer_mode": "spatial_template",
+ "warmup_epochs": 0,
+ "weight_decay": 0.0001,
+ "weight_scale": 1.0,
+ "width": 16
+ },
+ "calibration_metric_space": "reciprocal_local_activity_products",
+ "counters": {
+ "apical_warmup_examples": 0,
+ "calibration_event_examples": 0,
+ "causal_scalar_observations": 0,
+ "logical_batch_loss_queries": 0,
+ "mirror_conv_examples": 0,
+ "mirror_events": 0,
+ "mirror_readout_examples": 0,
+ "ordinary_examples": 900000,
+ "per_example_loss_terms": 0,
+ "perturbation_events": 0,
+ "perturbation_forward_examples": 0,
+ "predictor_warmup_examples": 0
+ },
+ "diagnostics": {
+ "early_third_mean": 0.8855825364589691,
+ "feedback_forward_cosine": [
+ 0.9264333248138428,
+ 0.9220218658447266,
+ 0.9038588404655457,
+ 0.8975453972816467,
+ 0.9213621616363525,
+ 0.8933338522911072,
+ 0.8915912508964539,
+ 0.9265133142471313,
+ 0.9061511754989624,
+ 0.8942975997924805,
+ 0.9047679901123047,
+ 0.8851061463356018,
+ 0.8851138353347778,
+ 0.9171422719955444,
+ 0.9053213000297546,
+ 0.8831733465194702,
+ 0.8767958879470825,
+ 0.7925058007240295,
+ 0.9866083860397339
+ ],
+ "feedback_forward_norm_ratio": [
+ 0.9885096549987793,
+ 1.0436758995056152,
+ 0.9917821884155273,
+ 1.058516263961792,
+ 1.0159763097763062,
+ 1.0456026792526245,
+ 1.008077621459961,
+ 1.0261797904968262,
+ 1.0106136798858643,
+ 1.0411553382873535,
+ 1.0132758617401123,
+ 1.0498096942901611,
+ 1.0225006341934204,
+ 1.0299837589263916,
+ 1.011872410774231,
+ 1.0414197444915771,
+ 1.0175153017044067,
+ 1.0444023609161377,
+ 1.090652585029602
+ ],
+ "feedback_forward_relative_error": [
+ 0.3815428614616394,
+ 0.4058021008968353,
+ 0.43677207827568054,
+ 0.46938687562942505,
+ 0.4000545144081116,
+ 0.47448983788490295,
+ 0.46758344769477844,
+ 0.3892383575439453,
+ 0.43566325306892395,
+ 0.4709555506706238,
+ 0.4395098090171814,
+ 0.49367427825927734,
+ 0.48523032665252686,
+ 0.4142261743545532,
+ 0.4378886818885803,
+ 0.49502187967300415,
+ 0.5010300874710083,
+ 0.6598381400108337,
+ 0.1934662163257599
+ ],
+ "innovation_negative_gradient_cosine": [
+ 0.8906962275505066,
+ 0.8764338493347168,
+ 0.8894688487052917,
+ 0.8657850027084351,
+ 0.8935482501983643,
+ 0.8975630402565002,
+ 0.9018489122390747,
+ 0.9003725051879883,
+ 0.9135095477104187,
+ 0.8963901996612549,
+ 0.9183286428451538,
+ 0.9015354514122009,
+ 0.9177113771438599,
+ 0.9407275915145874,
+ 0.9454699754714966,
+ 0.9456664323806763,
+ 0.9667372107505798,
+ 0.9770752787590027,
+ 0.9845573902130127
+ ],
+ "mean_feedback_forward_cosine": 0.9010338814635026,
+ "mean_feedback_forward_relative_error": 0.44480918269408376,
+ "normalization_state": "training_batch_stats_without_running_update",
+ "raw_negative_gradient_cosine": [
+ 0.8906962275505066,
+ 0.8764338493347168,
+ 0.8894688487052917,
+ 0.8657850027084351,
+ 0.8935482501983643,
+ 0.8975630402565002,
+ 0.9018489122390747,
+ 0.9003725051879883,
+ 0.9135095477104187,
+ 0.8963901996612549,
+ 0.9183286428451538,
+ 0.9015354514122009,
+ 0.9177113771438599,
+ 0.9407275915145874,
+ 0.9454699754714966,
+ 0.9456664323806763,
+ 0.9667372107505798,
+ 0.9770752787590027,
+ 0.9845573902130127
+ ],
+ "teaching_negative_gradient_cosine": [
+ 0.8906962275505066,
+ 0.8764338493347168,
+ 0.8894688487052917,
+ 0.8657850027084351,
+ 0.8935482501983643,
+ 0.8975630402565002,
+ 0.9018489122390747,
+ 0.9003725051879883,
+ 0.9135095477104187,
+ 0.8963901996612549,
+ 0.9183286428451538,
+ 0.9015354514122009,
+ 0.9177113771438599,
+ 0.9407275915145874,
+ 0.9454699754714966,
+ 0.9456664323806763,
+ 0.9667372107505798,
+ 0.9770752787590027,
+ 0.9845573902130127
+ ],
+ "wall_s": 0.1368110179901123
+ },
+ "epochs": [
+ {
+ "epoch": 1,
+ "feedback_tracking": {
+ "max_feedback_forward_relative_error": 1.319318175315857,
+ "mean_feedback_forward_cosine": 0.26153333798835154,
+ "mean_feedback_forward_relative_error": 1.2144870224751925,
+ "min_feedback_forward_cosine": 0.1322619915008545
+ },
+ "lr": 0.1,
+ "output_lr": 0.1,
+ "step": 352,
+ "train_examples": 45000,
+ "train_loss": 1.831467347378201
+ },
+ {
+ "epoch": 2,
+ "feedback_tracking": {
+ "max_feedback_forward_relative_error": 1.2504557371139526,
+ "mean_feedback_forward_cosine": 0.40932882773248774,
+ "mean_feedback_forward_relative_error": 1.0874902041334855,
+ "min_feedback_forward_cosine": 0.22793743014335632
+ },
+ "lr": 0.1,
+ "output_lr": 0.1,
+ "step": 704,
+ "train_examples": 45000,
+ "train_loss": 1.366863529311286
+ },
+ {
+ "epoch": 3,
+ "feedback_tracking": {
+ "max_feedback_forward_relative_error": 1.19105064868927,
+ "mean_feedback_forward_cosine": 0.5041392759272927,
+ "mean_feedback_forward_relative_error": 0.9971762236795927,
+ "min_feedback_forward_cosine": 0.30597037076950073
+ },
+ "lr": 0.1,
+ "output_lr": 0.1,
+ "step": 1056,
+ "train_examples": 45000,
+ "train_loss": 1.066310328886244
+ },
+ {
+ "epoch": 4,
+ "feedback_tracking": {
+ "max_feedback_forward_relative_error": 1.1481163501739502,
+ "mean_feedback_forward_cosine": 0.5715355308432328,
+ "mean_feedback_forward_relative_error": 0.9270231566931072,
+ "min_feedback_forward_cosine": 0.3590141832828522
+ },
+ "lr": 0.1,
+ "output_lr": 0.1,
+ "step": 1408,
+ "train_examples": 45000,
+ "train_loss": 0.8910715007040235
+ },
+ {
+ "epoch": 5,
+ "feedback_tracking": {
+ "max_feedback_forward_relative_error": 1.1075339317321777,
+ "mean_feedback_forward_cosine": 0.62295485170264,
+ "mean_feedback_forward_relative_error": 0.8701311349868774,
+ "min_feedback_forward_cosine": 0.40619510412216187
+ },
+ "lr": 0.1,
+ "output_lr": 0.1,
+ "step": 1760,
+ "train_examples": 45000,
+ "train_loss": 0.7831426847881741
+ },
+ {
+ "epoch": 6,
+ "feedback_tracking": {
+ "max_feedback_forward_relative_error": 1.0683236122131348,
+ "mean_feedback_forward_cosine": 0.6647130376414249,
+ "mean_feedback_forward_relative_error": 0.8211206464391005,
+ "min_feedback_forward_cosine": 0.44973599910736084
+ },
+ "lr": 0.1,
+ "output_lr": 0.1,
+ "step": 2112,
+ "train_examples": 45000,
+ "train_loss": 0.7067856804741753
+ },
+ {
+ "epoch": 7,
+ "feedback_tracking": {
+ "max_feedback_forward_relative_error": 1.0321979522705078,
+ "mean_feedback_forward_cosine": 0.6994027605182246,
+ "mean_feedback_forward_relative_error": 0.7776816581424913,
+ "min_feedback_forward_cosine": 0.487300843000412
+ },
+ "lr": 0.1,
+ "output_lr": 0.1,
+ "step": 2464,
+ "train_examples": 45000,
+ "train_loss": 0.6526998540454441
+ },
+ {
+ "epoch": 8,
+ "feedback_tracking": {
+ "max_feedback_forward_relative_error": 0.9973180890083313,
+ "mean_feedback_forward_cosine": 0.7283705755283958,
+ "mean_feedback_forward_relative_error": 0.7394031474464818,
+ "min_feedback_forward_cosine": 0.5232464075088501
+ },
+ "lr": 0.1,
+ "output_lr": 0.1,
+ "step": 2816,
+ "train_examples": 45000,
+ "train_loss": 0.6056941405296326
+ },
+ {
+ "epoch": 9,
+ "feedback_tracking": {
+ "max_feedback_forward_relative_error": 0.9644169211387634,
+ "mean_feedback_forward_cosine": 0.7529234384235582,
+ "mean_feedback_forward_relative_error": 0.705177787103151,
+ "min_feedback_forward_cosine": 0.5548381805419922
+ },
+ "lr": 0.1,
+ "output_lr": 0.1,
+ "step": 3168,
+ "train_examples": 45000,
+ "train_loss": 0.5672190690252517
+ },
+ {
+ "epoch": 10,
+ "feedback_tracking": {
+ "max_feedback_forward_relative_error": 0.9325018525123596,
+ "mean_feedback_forward_cosine": 0.7752530323831659,
+ "mean_feedback_forward_relative_error": 0.6724935864147387,
+ "min_feedback_forward_cosine": 0.5845517516136169
+ },
+ "lr": 0.1,
+ "output_lr": 0.1,
+ "step": 3520,
+ "train_examples": 45000,
+ "train_loss": 0.5412863729900784
+ },
+ {
+ "epoch": 11,
+ "feedback_tracking": {
+ "max_feedback_forward_relative_error": 0.9008499383926392,
+ "mean_feedback_forward_cosine": 0.7944865289487337,
+ "mean_feedback_forward_relative_error": 0.6431144601420352,
+ "min_feedback_forward_cosine": 0.6125654578208923
+ },
+ "lr": 0.1,
+ "output_lr": 0.1,
+ "step": 3872,
+ "train_examples": 45000,
+ "train_loss": 0.5144231326527066
+ },
+ {
+ "epoch": 12,
+ "feedback_tracking": {
+ "max_feedback_forward_relative_error": 0.870076596736908,
+ "mean_feedback_forward_cosine": 0.8117237467514841,
+ "mean_feedback_forward_relative_error": 0.6155103241142473,
+ "min_feedback_forward_cosine": 0.6389982104301453
+ },
+ "lr": 0.1,
+ "output_lr": 0.1,
+ "step": 4224,
+ "train_examples": 45000,
+ "train_loss": 0.4893017247200012
+ },
+ {
+ "epoch": 13,
+ "feedback_tracking": {
+ "max_feedback_forward_relative_error": 0.8421436548233032,
+ "mean_feedback_forward_cosine": 0.8270191926705209,
+ "mean_feedback_forward_relative_error": 0.5898554764295879,
+ "min_feedback_forward_cosine": 0.662228524684906
+ },
+ "lr": 0.1,
+ "output_lr": 0.1,
+ "step": 4576,
+ "train_examples": 45000,
+ "train_loss": 0.4666704408009847
+ },
+ {
+ "epoch": 14,
+ "feedback_tracking": {
+ "max_feedback_forward_relative_error": 0.8131178021430969,
+ "mean_feedback_forward_cosine": 0.8408146061395344,
+ "mean_feedback_forward_relative_error": 0.5656935533410624,
+ "min_feedback_forward_cosine": 0.6854491233825684
+ },
+ "lr": 0.1,
+ "output_lr": 0.1,
+ "step": 4928,
+ "train_examples": 45000,
+ "train_loss": 0.4554117670377096
+ },
+ {
+ "epoch": 15,
+ "feedback_tracking": {
+ "max_feedback_forward_relative_error": 0.7860043048858643,
+ "mean_feedback_forward_cosine": 0.8532820532196447,
+ "mean_feedback_forward_relative_error": 0.5429034452689322,
+ "min_feedback_forward_cosine": 0.7059769034385681
+ },
+ "lr": 0.1,
+ "output_lr": 0.1,
+ "step": 5280,
+ "train_examples": 45000,
+ "train_loss": 0.437559742252032
+ },
+ {
+ "epoch": 16,
+ "feedback_tracking": {
+ "max_feedback_forward_relative_error": 0.7597430944442749,
+ "mean_feedback_forward_cosine": 0.8647166176846153,
+ "mean_feedback_forward_relative_error": 0.5210003123471612,
+ "min_feedback_forward_cosine": 0.7249040603637695
+ },
+ "lr": 0.1,
+ "output_lr": 0.1,
+ "step": 5632,
+ "train_examples": 45000,
+ "train_loss": 0.4244806206067403
+ },
+ {
+ "epoch": 17,
+ "feedback_tracking": {
+ "max_feedback_forward_relative_error": 0.7334027886390686,
+ "mean_feedback_forward_cosine": 0.8751521863435444,
+ "mean_feedback_forward_relative_error": 0.5003075152635574,
+ "min_feedback_forward_cosine": 0.7437110543251038
+ },
+ "lr": 0.1,
+ "output_lr": 0.1,
+ "step": 5984,
+ "train_examples": 45000,
+ "train_loss": 0.4090740834448073
+ },
+ {
+ "epoch": 18,
+ "feedback_tracking": {
+ "max_feedback_forward_relative_error": 0.7081427574157715,
+ "mean_feedback_forward_cosine": 0.8844657948142604,
+ "mean_feedback_forward_relative_error": 0.4810000712934293,
+ "min_feedback_forward_cosine": 0.7609646320343018
+ },
+ "lr": 0.1,
+ "output_lr": 0.1,
+ "step": 6336,
+ "train_examples": 45000,
+ "train_loss": 0.39670369214481777
+ },
+ {
+ "epoch": 19,
+ "feedback_tracking": {
+ "max_feedback_forward_relative_error": 0.6829538345336914,
+ "mean_feedback_forward_cosine": 0.8932415654784754,
+ "mean_feedback_forward_relative_error": 0.4622263304497066,
+ "min_feedback_forward_cosine": 0.7776626348495483
+ },
+ "lr": 0.1,
+ "output_lr": 0.1,
+ "step": 6688,
+ "train_examples": 45000,
+ "train_loss": 0.3911194394800398
+ },
+ {
+ "epoch": 20,
+ "feedback_tracking": {
+ "max_feedback_forward_relative_error": 0.6598381400108337,
+ "mean_feedback_forward_cosine": 0.9010338814635026,
+ "mean_feedback_forward_relative_error": 0.44480918269408376,
+ "min_feedback_forward_cosine": 0.7925058007240295
+ },
+ "lr": 0.1,
+ "output_lr": 0.1,
+ "step": 7040,
+ "train_examples": 45000,
+ "train_loss": 0.3764604419231415
+ }
+ ],
+ "evaluation_protocol": {
+ "test_evaluations": 0,
+ "test_used_for_selection": false,
+ "validation_evaluations": 1
+ },
+ "final": {
+ "accuracy": 0.8266,
+ "epoch": 20,
+ "evaluation_split": "validation",
+ "finite": true,
+ "loss": 0.5224007934570313,
+ "step": 7040
+ },
+ "hardware": {
+ "cuda_device_name": "NVIDIA GeForce GTX 1080",
+ "cuda_visible_devices": "7",
+ "device": "cuda",
+ "device_total_memory_bytes": 8507949056,
+ "peak_memory_allocated_bytes": 2162790400,
+ "peak_memory_reserved_bytes": 2866806784,
+ "torch_version": "2.3.1+cu118"
+ },
+ "protocol_family": "oral_a_cifar_local_resnet_development",
+ "provenance": {
+ "git_commit": "8f420f256aef160d79e3a9620a92ae156f06ef74",
+ "git_tracked_dirty": false
+ },
+ "schema_version": 1,
+ "split": {
+ "cifar_source_files": [
+ {
+ "bytes": 31035704,
+ "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_1",
+ "sha256": "54636561a3ce25bd3e19253c6b0d8538147b0ae398331ac4a2d86c6d987368cd"
+ },
+ {
+ "bytes": 31035320,
+ "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_2",
+ "sha256": "766b2cef9fbc745cf056b3152224f7cf77163b330ea9a15f9392beb8b89bc5a8"
+ },
+ {
+ "bytes": 31035999,
+ "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_3",
+ "sha256": "0f00d98ebfb30b3ec0ad19f9756dc2630b89003e10525f5e148445e82aa6a1f9"
+ },
+ {
+ "bytes": 31035696,
+ "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_4",
+ "sha256": "3f7bb240661948b8f4d53e36ec720d8306f5668bd0071dcb4e6c947f78e9682b"
+ },
+ {
+ "bytes": 31035623,
+ "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/data_batch_5",
+ "sha256": "d91802434d8376bbaeeadf58a737e3a1b12ac839077e931237e0dcd43adcb154"
+ },
+ {
+ "bytes": 31035526,
+ "path": "/home/yurenh2/sdrn/data/cifar-10-batches-py/test_batch",
+ "sha256": "f53d8d457504f7cff4ea9e021afcf0e0ad8e24a91f3fc42091b8adef61157831"
+ }
+ ],
+ "dataset": "cifar10",
+ "evaluation_split": "validation",
+ "input_layout": "NCHW",
+ "input_shape": [
+ 3,
+ 32,
+ 32
+ ],
+ "loader_seed": 0,
+ "normalization_mean": [
+ 0.49140000343322754,
+ 0.4821999967098236,
+ 0.4465000033378601
+ ],
+ "normalization_std": [
+ 0.24699999392032623,
+ 0.2434999942779541,
+ 0.26159998774528503
+ ],
+ "split_from_training_only": true,
+ "split_seed": 2027,
+ "test_examples": 10000,
+ "train_examples": 45000,
+ "training_augmentation": "random_crop_32_padding_4_zero_then_horizontal_flip_p0.5",
+ "validation_class_counts": {
+ "0": 500,
+ "1": 500,
+ "2": 500,
+ "3": 500,
+ "4": 500,
+ "5": 500,
+ "6": 500,
+ "7": 500,
+ "8": 500,
+ "9": 500
+ },
+ "validation_examples": 5000,
+ "validation_index_sha256": "8328b206a97c420e49e54e3eca4abe3274c4756b084355784ea3fb8059e4515b"
+ },
+ "timing": {
+ "apical_warmup_wall_s": 0.0,
+ "evaluation_wall_s": 0.36439943313598633,
+ "mirror_warmup_wall_s": 0.0,
+ "predictor_warmup_wall_s": 0.0,
+ "timing_excludes_data_loading_hashing_and_model_construction": true,
+ "total_timed_wall_s": 558.1345100402832,
+ "train_wall_s": 557.5914545059204
+ },
+ "work": {
+ "apical_macs_per_example": 40108672,
+ "causal_scalar_observations": 0,
+ "components": {
+ "apical_projection_macs": 36097804800000,
+ "apical_regression_macs": 0,
+ "bp_reverse_macs_estimate": 0,
+ "kp_reciprocal_correlation_macs": 36097804800000,
+ "local_weight_correlation_macs": 36495936000000,
+ "mirror_feedback_prediction_macs": 0,
+ "mirror_local_correlation_macs": 0,
+ "mirror_response_macs": 0,
+ "ordinary_forward_macs": 36495936000000,
+ "perturbation_forward_macs": 0,
+ "warmup_clean_forward_macs": 0
+ },
+ "definition": "multiply-accumulates in conv/linear maps; one local weight correlation equals one forward-weight MAC count; BP reverse is estimated as one weight-gradient plus one activation-gradient convolution per forward convolution; elementwise nonlinearities and optimizer arithmetic excluded",
+ "forward_macs_per_example": 40551040,
+ "logical_batch_loss_queries": 0,
+ "mirror_probe_examples": 0,
+ "mirror_readout_probe_examples": 0,
+ "per_example_cross_entropy_terms": 0,
+ "total_clean_forward_examples": 900000,
+ "total_forward_equivalent_examples": 900000,
+ "total_macs_estimate": 145187481600000
+ }
+}
diff --git a/results/kp_short_gate.json b/results/kp_short_gate.json
new file mode 100644
index 0000000..22e5b27
--- /dev/null
+++ b/results/kp_short_gate.json
@@ -0,0 +1,35 @@
+{
+ "checks": {
+ "accuracy_at_least_0.70": true,
+ "early_alignment_at_least_0.50": true,
+ "epoch11_to20_feedback_cosine_at_least_0.70": true,
+ "final_feedback_cosine_at_least_0.80": true,
+ "macs_at_most_1.40x_bp": true,
+ "record_and_trajectory_finite": true,
+ "within_15_points_of_bp_epoch20": true,
+ "zero_task_loss_queries": true
+ },
+ "confirmation_test_seeds_touched": false,
+ "full_validation_opened": true,
+ "metrics": {
+ "accuracy": 0.8266,
+ "bp_epoch20_accuracy": 0.8102,
+ "bp_epoch20_total_macs": 109487808000000,
+ "early_third_alignment": 0.8855825364589691,
+ "epoch11_to20_mean_feedback_forward_cosine": 0.8545936173514315,
+ "final_mean_feedback_forward_cosine": 0.9010338814635026,
+ "final_mean_feedback_forward_relative_error": 0.44480918269408376,
+ "logical_batch_loss_queries": 0,
+ "loss": 0.5224007934570313,
+ "mac_ratio_to_bp": 1.3260607208430002,
+ "peak_memory_allocated_bytes": 2162790400,
+ "source_commit": "8f420f256aef160d79e3a9620a92ae156f06ef74",
+ "total_macs": 145187481600000,
+ "wall_s": 558.1345100402832
+ },
+ "protocol": "kolen_pollack_short_v1",
+ "review_score_after": 5,
+ "review_score_before": 5,
+ "score_change_rule": "inherited KP baseline cannot raise score",
+ "status": "passed"
+}