diff options
| -rw-r--r-- | BASELINES.md | 5 | ||||
| -rw-r--r-- | KP_BASELINE.md | 26 | ||||
| -rw-r--r-- | README.md | 14 | ||||
| -rw-r--r-- | RESULTS.md | 14 | ||||
| -rw-r--r-- | REVIEW_SCORECARD.md | 2 | ||||
| -rw-r--r-- | ROADMAP.md | 14 |
6 files changed, 64 insertions, 11 deletions
diff --git a/BASELINES.md b/BASELINES.md index 5c66e2a..760a031 100644 --- a/BASELINES.md +++ b/BASELINES.md @@ -41,7 +41,10 @@ The audited implementations and completed results are in `RESULTS.md`: reciprocal correlation is charged in the MAC audit; `KP_BASELINE.md` freezes its staged ResNet gate before any KP task endpoint. The frozen KP-1 record passes at 82.66% validation accuracy versus matched BP's epoch-20 81.02%, - with 0.886 early alignment and 1.326x BP MACs; + with 0.886 early alignment and 1.326x BP MACs. The frozen KP-2 record also + passes at 91.26% versus matched BP's 91.62%, with 0.9994 early alignment, + 0.9997 late feedback cosine, zero loss queries, and the same 1.326x MAC + ratio; - direct node perturbation uses a causal target on every hidden update and no learned vectorizer; - the no-traffic, `P=0` SDIL backbone is learned direct feedback by node diff --git a/KP_BASELINE.md b/KP_BASELINE.md index 78bb756..27fd5ff 100644 --- a/KP_BASELINE.md +++ b/KP_BASELINE.md @@ -96,4 +96,28 @@ All epoch losses and tracking diagnostics are finite. The run uses require exact weight equality to obtain a strong teaching direction. Unlike the failed intermittent-mirror trajectory, these alignment gates cover the constant-LR training interval rather than only a decayed-LR endpoint. KP-2 is -opened without changing any optimizer or feedback setting. +opened without changing any optimizer or feedback setting and subsequently +passes at those exact settings. + +## Audited KP-2 outcome + +The sole frozen full record from source revision `44b110f` passes all seven +gates: + +| metric | KP-2 | frozen requirement/reference | +|:--|--:|--:| +| epoch-200 validation accuracy | **91.26%** | at least 88%; matched BP is 91.62% | +| validation loss | 0.429013 | finite | +| early-third teaching alignment | 0.999397 | at least 0.80 | +| final feedback/forward cosine | 0.999663 | at least 0.95 | +| epoch-151--200 mean feedback cosine | 0.999663 | at least 0.95 | +| final feedback/forward relative error | 0.025458 | diagnostic | +| MAC ratio to matched BP | 1.3261x | at most 1.40x | +| task-loss queries for feedback | 0 | exactly 0 | + +Every epoch loss and tracking diagnostic is finite. The run uses +`1.451874816e15` MACs, 2.163 GB peak allocated memory, and 5579.7 s on a GTX +1080. It is 0.36 accuracy points below matched BP while using no reverse-mode +gradient or loss query to train its feedback path. The result opens MT-1 but +does not change the 5/10 reviewer score: reciprocal KP is inherited substrate, +not evidence that somato-dendritic innovation is load-bearing. @@ -58,7 +58,8 @@ and scaling behavior. See `NOVELTY.md` for the exact prior-art boundary. run at 10% with NaN validation loss. Its endpoint Q/W cosine nevertheless reaches 0.999998, exposing final alignment as an inadequate certificate for an intermittent tracker. The separate reciprocal KP short gate reaches - 82.66% and remains the active strong substrate. + 82.66%, and its frozen full gate reaches 91.26% versus matched BP's 91.62% + with 0.9997 late feedback cosine. It is the active strong substrate. - Native author-code fidelity is complete. BurstCCN reaches `80.10%` at its validation-selected epoch versus published `82.97 +/- 0.21%`; Dual Prop reaches `92.46%` versus published `92.41 +/- 0.07%`. Their audited walls are @@ -156,11 +157,12 @@ no SDIL novelty credit. `RESIDUAL_MIRROR.md` freezes its capture and conditional task gates. The stronger reciprocal Kolen--Pollack substrate is audited separately in -`KP_BASELINE.md`. Its short gate passed, while its full frozen baseline is in -progress. Before that full endpoint, `MIXED_TRAFFIC.md` froze the actual -Harnett-specific test: raw, norm-matched raw, and innovation under identical -four-times-RMS soma-predictable apical traffic and predictor cost. Mechanics -are green; task access remains conditional on the KP full gate. +`KP_BASELINE.md`. Its short and full frozen gates pass; the full record reaches +91.26% validation accuracy, 0.9994 early alignment, and 1.326x BP MACs without +feedback loss queries. Before that full endpoint, `MIXED_TRAFFIC.md` froze the +actual Harnett-specific test: raw, norm-matched raw, and innovation under +identical four-times-RMS soma-predictable apical traffic and predictor cost. +Mechanics are green and MT-1 task access is now open. The subsequent V3 mechanism estimates the required A/G matrix statistics directly by perturbing the vectorizer parameter subspace. It remains @@ -909,6 +909,20 @@ correlation, and remains finite throughout the constant-LR screen. The result opens the frozen KP-2 full baseline. It is entirely prior-art baseline evidence and does not change the SDIL reviewer score. +The sole frozen 200-epoch KP-2 record also passes every gate: + +| method | validation accuracy | early alignment | epoch-151--200 feedback cosine | estimated MACs | +|:--|--:|--:|--:|--:| +| KP | **91.26%** | 0.999397 | 0.999663 | 1.4519e15 | +| matched BP | 91.62% | n/a | exact by construction | 1.0949e15 | + +KP-2 ends with finite validation loss `0.429013`, final feedback/forward cosine +`0.999663`, zero task-loss queries, and `1.3261x` BP MACs. Peak allocated +memory is 2.163 GB and audited wall time is 5579.7 s on a GTX 1080. This +establishes a stable, near-BP standard-ResNet substrate and opens the frozen +mixed-traffic screen. It remains prior-art evidence and therefore leaves the +reviewer score at 5/10 until innovation itself wins the controlled ablation. + ## How to run `experiments/run.py --mode {bp,fa,dfa,sdil} --dataset {mnist,fmnist,cifar10} --depth D --residual {0,1} --act {tanh,gelu,silu,relu}` Batteries: `experiments/run_v2.sh <ds> "<depths>" <res> <act> "<seeds>" <ep> <pfx>`. diff --git a/REVIEW_SCORECARD.md b/REVIEW_SCORECARD.md index bf08e18..73fb424 100644 --- a/REVIEW_SCORECARD.md +++ b/REVIEW_SCORECARD.md @@ -68,6 +68,7 @@ Every formal result report records: | Residual response mirror short gate passes | 5 | RRM reaches 65.08%, within 9.86 points of matched BP, at 0.9969x BP MACs and nearly exact alignment | Opens one full validation run; the inherited baseline still supplies no Harnett-specific evidence | | Residual response mirror full gate fails | 5 | RRM ends at 10% with NaN validation loss although endpoint Q/W cosine is 0.999998 | Closes intermittent mirroring and proves endpoint alignment is an inadequate trajectory certificate; KP remains the separate strong substrate | | Modified KP short gate passes | 5 | KP reaches 82.66% versus BP's epoch-20 81.02%, with 0.886 early alignment and 1.326x BP MACs | Strongly solves the substrate at useful scale, but all credit belongs to inherited reciprocal plasticity until innovation is load-bearing | +| Modified KP full gate passes | 5 | KP reaches 91.26% versus matched BP's 91.62%, with 0.9994 early alignment, 0.9997 late feedback cosine, zero queries, and 1.326x BP MACs | Establishes a stable near-BP ResNet substrate and opens MT-1, but inherited KP evidence cannot raise the SDIL score | | Mixed-traffic MT-1 | 5 if passed | A frozen short raw/matched/innovation screen must preserve clean-KP utility and show accuracy plus same-state directional gains | One seed and 20 epochs can only open full validation | | Mixed-traffic MT-2 | 5 if passed | A full seed-0 panel must reach 88%, stay within 3 points of clean KP, and beat both controls under audited cost | A single development seed still cannot establish acceptance | | Mixed-traffic MT-3 | 6 if passed | Five untouched network/traffic draws must retain 88% mean test accuracy and positive confidence-bounded paired gains | Establishes the controlled ResNet innovation claim; natural traffic and broad depth/architecture scaling remain open | @@ -97,6 +98,7 @@ retroactively reopened by success on standard vision benchmarks. | 2026-07-22 / residual response-mirror RRM-2 | Two clean short ResNet records; selected RRM reaches 65.08%, 9.86 points below BP, with 0.9991 early alignment and 0.9969x BP MACs | 5 → 5 | Narrowly opens the full baseline run and strengthens the efficient comparator; the remaining accuracy gap despite near-exact direction warns that alignment is not trajectory equivalence | | 2026-07-22 / residual response-mirror RRM-3 | The sole frozen full record reaches only 10%, has NaN validation loss, and peaks at `1.5808e16` training loss despite 0.999998 endpoint Q/W cosine | 5 → 5 | Closes intermittent residual mirroring and turns alignment latency into a measured negative; it neither weakens the surviving SDIL mechanism claim nor supplies positive scale evidence | | 2026-07-22 / modified KP-1 | One clean constant-LR ResNet record reaches 82.66%, above BP's epoch-20 81.02%, with 0.8856 early alignment, 0.8546 train-period feedback cosine, zero queries, and 1.326x BP MACs | 5 → 5 | Establishes a stable strong substrate under a gate that observes the training trajectory, but reciprocal Kolen--Pollack plasticity is prior art and adds no Harnett-specific evidence | +| 2026-07-22 / `2ef7f94` modified KP-2 | One clean full ResNet record reaches 91.26%, 0.36 points below matched BP, with 0.9994 early alignment, 0.9997 late feedback cosine, zero queries, and 1.326x BP MACs | 5 → 5 | Removes substrate stability as the immediate blocker and opens the load-bearing innovation experiment, but prior-art reciprocal plasticity earns no novelty credit | | 2026-07-22 / mixed-traffic MT-0 | Exact raw/matched/innovation mechanics pass; synthetic ResNet-20 matched batch peaks at 0.857 GB allocated on GTX 1080 without task data | 5 → 5 | Removes implementation, graph, and memory objections before endpoint access; supplies no evidence yet that innovation is useful | Future rows are appended only after an audited frozen stage. A score staying flat is informative: @@ -455,8 +455,8 @@ cosine `0.999998`, and norm ratios `[1.0002,1.0024]`. This closes RRM without cadence/rate rescue and demonstrates why endpoint cosine cannot substitute for a training-trajectory audit. No innovation or test panel opens from RRM. -**Modified Kolen--Pollack baseline protocol: mechanics passed, task gate -frozen.** The separate local reciprocal correlations, post-observation W/Q +**Modified Kolen--Pollack baseline protocol: mechanics and both task gates +passed.** The separate local reciprocal correlations, post-observation W/Q independence, and exact symmetric-limit momentum updates all pass at zero numerical error. `KP_BASELINE.md` freezes one 20-epoch full-development screen with no LR/decay grid and requires both validation accuracy and training-period @@ -473,6 +473,14 @@ the frozen settings. This strongly repairs the feedback-substrate engineering problem but remains inherited Akrout et al. evidence, so the paper score stays 5/10. +**Modified Kolen--Pollack KP-2 status: passed.** The sole frozen 200-epoch +record reaches `91.26%` validation accuracy versus matched BP's `91.62%`. +Early alignment is `0.999397`; final and epoch-151--200 mean feedback/forward +cosines are both `0.999663`. The complete trajectory is finite, feedback uses +zero task-loss queries, and cost remains `1.3261x` BP. This opens MT-1 without +raising the reviewer score because the result is entirely inherited KP +evidence. + **KP mixed-traffic MT-0 status: mechanics passed; MT-1--MT-3 frozen before endpoints.** `MIXED_TRAFFIC.md` fixes a four-to-one, initialization-calibrated soma-predictable traffic intervention and crosses raw apical activity, @@ -486,7 +494,7 @@ MACs, and every control pays the predictor schedule. The synthetic batch-128 matched path is finite on a GTX 1080 and peaks at 0.857 GB allocated after reset; no task endpoint was used for this hardware check. -MT-1 remains sealed until KP-2 passes. A 20-epoch pass opens one full seed-0 +MT-1 is now open after the KP-2 pass. A 20-epoch pass opens one full seed-0 validation panel but cannot raise the reviewer score; a full pass opens the already frozen five-seed, all-50k, one-test-evaluation confirmation. Only the complete MT-3 confirmation can move the score from 5/10 to 6/10. This is the |